Some days ago we installed Istio in Kubernetes Multi-Cluster. Now it's time for some Day 2 Operations with Kiali, Jaeger, Grafana, and Prometheus.
Prerequisites: Istio Multi-Cluster running in Rancher
[Kiali](https://kiali.io/) is a tool to manage, visualize, validate and troubleshoot
your Service Mesh. Kiali provides like Istio 3 install methods:
Install by Rancher as sub chart of rancher-istio
Install by origin Kiali Helm Chart
Install by Kiali Operator
Kiali is in heavy developement and with Rancher provided versions have
[bugs](https://github.com/kiali/kiali/issues/4132). For this reason we use the Rancher
Helm chart but latest version for installation on bothe clusters:
$ cat << eof> values-kiali.yaml
# Specify rancher clusterId of external tracing config
Within the provided credentials for remote Istio installations Kiali
will discover this endpoints as well. But multi-cluster feature is in
experimental state, don't expect too much. A very tiny bug is the API
endpoint port for the remote cluster. Read [this discussion](https://github.com/kiali/kiali/discussions/4595)
how to handle that.
Addionally we need NetworkPolicy to communicate with Istio, Jaeger, KubeAPI,
and Prometheus. Beware that ServiceEndpoints sometimes are on different port
as Service ports!
$ cat <<eof | kubectl -n istio-system apply -f -
- port: 8080
- port: 8081
- port: 9090
- port: 16686
- port: 16686
- port: 8080
[Jaeger](https://www.jaegertracing.io/) monitor and troubleshoot transaction in
distributed systems. Jaeger is a very powerful tool which can configure with
Cassandra, Elasticsearch or Kafka Cluster. It would be an extra project to
install Jaeger with such kind of sizing. There is also a "all-in-one" installation
which will we use here.
[https://raseed-test.external.otc.telekomcloud.com/k8s/clusters/c-pzk8b/api/v1/namespaces/istio-system/services/http:kiali:20001/proxy/](<a href="https://raseed-test.external.otc.telekomcloud.com/k8s/clusters/c-pzk8b/api/v1/namespaces/istio-system/services/http:kiali:20001/proxy/" class="linebreak">https://raseed-test.external.otc.telekomcloud.com/k8s/clusters/c-pzk8b/api/v1/namespaces/istio-system/services/http:kiali:20001/proxy/</a>) on both cluster.</p>
<p>Not very common. Rancher 2.6 provides NavLinks resources to extend the<br />
Kiali verified your Istio installation, shows problems and errors. At the end traffic flows
are shown in graphs, requests are listed in details.
Data are provided by Jaeger which logs requests and network traffic in detail,
based on the source interface and the commands.
In Istio are already endpoints provided for scraping metrics.
All what we need are network access (already provided in
NetworkPolicy) and ServiceMonitor (when Prometheus Operator is
installed from rancher-monitoring)