Skip to content

Latest commit

 

History

History
116 lines (89 loc) · 4.65 KB

installing-logging-metrics-traces.md

File metadata and controls

116 lines (89 loc) · 4.65 KB

Monitoring, Logging and Tracing Installation

Knative Serving offers two different monitoring setups: Elasticsearch, Kibana, Prometheus and Grafana or Stackdriver, Prometheus and Grafana. You can install only one of these two setups and side-by-side installation of these two are not supported.

Elasticsearch, Kibana, Prometheus & Grafana Setup

If you installed the full Knative release, skip this step and continue to Create Elasticsearch Indices

  • Install Knative monitoring components from the root of the Serving repository:

    kubectl apply -R -f config/monitoring/100-common \
    -f config/monitoring/150-elasticsearch \
    -f third_party/config/monitoring/common \
    -f third_party/config/monitoring/elasticsearch \
    -f config/monitoring/200-common \
    -f config/monitoring/200-common/100-istio.yaml

  • The installation is complete when logging & monitoring components are all reported Running or Completed:

    kubectl get pods -n monitoring --watch
    NAME                                  READY     STATUS    RESTARTS   AGE
elasticsearch-logging-0               1/1       Running   0          2d
elasticsearch-logging-1               1/1       Running   0          2d
fluentd-ds-5kc85                      1/1       Running   0          2d
fluentd-ds-vhrcq                      1/1       Running   0          2d
fluentd-ds-xghk9                      1/1       Running   0          2d
grafana-798cf569ff-v4q74              1/1       Running   0          2d
kibana-logging-7d474fbb45-6qb8x       1/1       Running   0          2d
kube-state-metrics-75bd4f5b8b-8t2h2   4/4       Running   0          2d
node-exporter-cr6bh                   2/2       Running   0          2d
node-exporter-mf6k7                   2/2       Running   0          2d
node-exporter-rhzr7                   2/2       Running   0          2d
prometheus-system-0                   1/1       Running   0          2d
prometheus-system-1                   1/1       Running   0          2d

    CTRL+C to exit watch.

Create Elasticsearch Indices

To visualize logs with Kibana, you need to set which Elasticsearch indices to explore. We will create two indices in Elasticsearch using Logstash for application logs and Zipkin for request traces.

  • To open the Kibana UI (the visualization tool for Elasticsearch), start a local proxy with the following command:

    kubectl proxy

    This command starts a local proxy of Kibana on port 8001. For security reasons, the Kibana UI is exposed only within the cluster.

  • Navigate to the Kibana UI. It might take a couple of minutes for the proxy to work.

  • Within the "Configure an index pattern" page, enter logstash-* to Index pattern and select @timestamp from Time Filter field name and click on Create button.

Create logstash-* index

  • To create the second index, select Create Index Pattern button on top left of the page. Enter zipkin* to Index pattern and select timestamp_millis from Time Filter field name and click on Create button.

Stackdriver, Prometheus & Grafana Setup

If your Knative Serving is not built on a Google Cloud Platform (GCP) based cluster or you want to send logs to another GCP project, you need to build your own Fluentd image and modify the configuration first. See

  1. Install Fluentd image on Knative Serving.
  2. Set up a logging plugin.
  3. Install Knative monitoring components:
kubectl apply -R -f config/monitoring/100-common \
    -f config/monitoring/150-stackdriver-prod \
    -f third_party/config/monitoring/common \
    -f config/monitoring/200-common \
    -f config/monitoring/200-common/100-istio.yaml

Learn More

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License.