---
gitea: none
include_toc: true
---


# Logs

> **Note:**\
> Remember that you can use the command `watch` or `watch -n 5` (where 5 refers every 5 seconds) in case of being interested on execute this commands periodically.

## Istiod

```shell
kubectl logs -n istio-system -f deployments/istiod
```

## Istio-Proxy Pod

This will display the logs from a deployment while targeting the `istio-proxy` container from the targeted pod/deployment.

As well will attach the session to stream new logs. (`-f` `--follow`)

```shell
kubectl logs deployments/helloworld-default -f -c istio-proxy
```

```text
[2023-05-15T00:42:03.699Z] "- - -" 0 UH - - "-" 0 0 0 - "-" "-" "-" "-" "-" BlackHoleCluster - 10.111.90.232:8080 172.17.121.65:52006 - -
[2023-05-15T00:42:24.785Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 2 1 "-" "curl/7.74.0" "c133cbf0-b57d-4fba-8f84-d683ab903399" "helloworld.default.svc.cluster.local" "172.17.121.65:80" inbound|80|| 127.0.0.6:51695 172.17.121.65:80 172.17.121.65:43786 outbound_.80_._.helloworld.default.svc.cluster.local default
[2023-05-15T00:42:24.784Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 5 4 "-" "curl/7.74.0" "c133cbf0-b57d-4fba-8f84-d683ab903399" "helloworld.default.svc.cluster.local" "172.17.121.65:80" outbound|80||helloworld.default.svc.cluster.local 172.17.121.65:43786 10.111.90.232:80 172.17.121.65:57030 - default
[2023-05-15T00:43:23.209Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 6 5 "-" "curl/7.74.0" "e1f0a2f3-93ff-4c41-8cb3-6d3a53fce065" "helloworld.foo.svc.cluster.local" "172.17.247.42:80" outbound|80||helloworld.foo.svc.cluster.local 172.17.121.65:55040 10.109.248.148:80 172.17.121.65:60520 - default
[2023-05-15T00:43:29.751Z] "- - -" 0 UH - - "-" 0 0 0 - "-" "-" "-" "-" "-" BlackHoleCluster - 10.109.248.148:8080 172.17.121.65:40370 - -
[2023-05-15T00:43:31.979Z] "- - -" 0 UH - - "-" 0 0 0 - "-" "-" "-" "-" "-" BlackHoleCluster - 10.109.248.148:8080 172.17.121.65:40402 - -
```

## Ingress

The service targeted, `istio-ingressgateway`, is an Ingress Load Balancer service from Istio.

```shell
kubectl logs -n istio-system services/istio-ingressgateway
```
#### Invalid TLS context has neither subject CN nor SAN names

The TLS certificate specified don't have the field CN or the field SAN.

To address this issue, issue a new certificate that has at least one of those fields.

#### initial fetch timed out for type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.Secretthread

This is due not being able to retrieve the TLS configuration assigned to the gateway.

It's Important that the secret is located in the same namespace as the Istio Load Balancer used. In my case is the `istio-system`, but it will vary based on the environment.








# Istioctl analyze

`istioctl analyze` reviews the current configuration set.

Can be helpful to spot some improvements on the current configurations set, as well of the possibility of displaying misconfigurations / lack of them that might be causing issues.

```shell
istioctl analyze
```
```text
✔ No validation issues found when analyzing namespace: default.
```

By using the flag -A, it will review from all namespaces

```shell
istioctl analyze -A
```
```text
Info [IST0102] (Namespace istio-operator) The namespace is not enabled for Istio injection. Run 'kubectl label namespace istio-operator istio-injection=enabled' to enable it, or 'kubectl label namespace istio-operator istio-injection=disabled' to explicitly mark it as not needing injection.
Info [IST0118] (Service istio-system/grafana) Port name service (port: 3000, targetPort: 3000) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service istio-system/jaeger-collector) Port name jaeger-collector-grpc (port: 14250, targetPort: 14250) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service istio-system/jaeger-collector) Port name jaeger-collector-http (port: 14268, targetPort: 14268) doesn't follow the naming convention of Istio port.
```

One can specify/target a single namespace by using the flag `-n`

```shell
istioctl analyze -n istio-operator
```
```text
Info [IST0102] (Namespace istio-operator) The namespace is not enabled for Istio injection. Run 'kubectl label namespace istio-operator istio-injection=enabled' to enable it, or 'kubectl label namespace istio-operator istio-injection=disabled' to explicitly mark it as not needing injection.
```

## Example of spotting a misconfiguration

In this example, I have configured the gateway to listen to a port that currently is not open in the Isito Load Balancer selected.

```shell
istioctl analyze
```
```text
Warning [IST0104] (Gateway default/helloworld-gateway) The gateway refers to a port that is not exposed on the workload (pod selector istio=ingressgateway; port 81)
```

# Start the packet capture process on the istio-proxy container from a pod.

Target a pod and start a packet capture on the istio-proxy container.

This step requires Istio to be installed with the flag `values.global.proxy.privileged=true`

This is very useful to confirm if the service is receiving any traffic, or which is the traffic received.

If mTLS is  enabled and configured, the traffic received should be encrypted.

```shell
kubectl exec -n default  "$(kubectl get pod -n default -l app=helloworld -o jsonpath={.items..metadata.name})" -c istio-proxy -- sudo tcpdump dst port 80  -A
```
```text
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
...
```

# Istioctl proxy-status
> **Note:** Shorthand is `ps`

```shell
istioctl ps
```
```text
NAME                                                  CLUSTER        CDS        LDS        EDS        RDS          ECDS         ISTIOD                    VERSION
helloworld-6798765f88-ql26n.default                   Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-dbf5ff64-9kxxs     1.17.2
helloworld2-dc9cb5db6-m47x7.default                   Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-dbf5ff64-9kxxs     1.17.2
istio-egressgateway-676bf68b54-d28fn.istio-system     Kubernetes     SYNCED     SYNCED     SYNCED     NOT SENT     NOT SENT     istiod-dbf5ff64-9kxxs     1.17.2
istio-ingressgateway-8d56c999d-nv7ph.istio-system     Kubernetes     SYNCED     SYNCED     SYNCED     SYNCED       NOT SENT     istiod-dbf5ff64-9kxxs     1.17.2
```


# Istioctl proxy-config
> **Note:** Shorthand is `pc`


## Check listeners

Useful to review which is the configuration assigned to an Istio ingress. / Confirm if the configuration we are intending to deploy is being applied / learned.

### Get Istio ingress pod name

> **Note:**\
> Depending on the ingress gateway set, and your environment, it could be that the Load Balancer is not located in the namespace `istio-system`.

```shell
kubectl get pods -n istio-system               
```

```text
NAME                                    READY   STATUS    RESTARTS   AGE
grafana-6cb5b7fbb8-2nlp6                1/1     Running   0          2d3h
istio-ingressgateway-864db96c47-nvjc7   1/1     Running   0          20h
istiod-649d466b9-bwx7j                  1/1     Running   0          2d8h
jaeger-cc4688b98-h52xt                  1/1     Running   0          2d3h
kiali-594965b98c-zc67p                  1/1     Running   0          2d3h
prometheus-67f6764db9-szd5b             2/2     Running   0          2d3h

```

### List listeners

```shell
kubectl get pods -n istio-system istio-ingressgateway-864db96c47-nvjc7
```

```text
istioctl proxy-config listeners -n istio-system istio-ingressgateway-864db96c47-nvjc7
ADDRESS PORT  MATCH       DESTINATION
0.0.0.0 8443  SNI: lb.net Route: https.443.secure-http.helloworld-gateway.default
0.0.0.0 15021 ALL         Inline Route: /healthz/ready*
0.0.0.0 15090 ALL         Inline Route: /stats/prometheus*
```

This makes reference to the configuration set in the gateway resources.
Here we can notice a route with SNI match "lb.net", which is listening to the port 443 and HTTPS protocol.

## Check logs verbosity level settings

`istioctl proxy-config log` will display the verbosity level set from each log type for the specified pod.

```shell
istioctl proxy-config log helloworld-nginx-5d99f88767-cwcmd 
```
```text
helloworld-nginx-5d99f88767-cwcmd.default:
active loggers:
  admin: warning
  alternate_protocols_cache: warning
  aws: warning
  assert: warning
  backtrace: warning
  cache_filter: warning
  client: warning
  config: warning
  connection: warning
...
```

## List all

It displays ALL from the specified pod.

```shell
istioctl proxy-config all helloworld-nginx-5d99f88767-cwcmd
```
```txt
SERVICE FQDN                                               PORT      SUBSET     DIRECTION     TYPE             DESTINATION RULE
                                                           80        -          inbound       ORIGINAL_DST     
BlackHoleCluster                                           -         -          -             STATIC           
InboundPassthroughClusterIpv4                              -         -          -             ORIGINAL_DST     
PassthroughCluster                                         -         -          -             ORIGINAL_DST     
agent                                                      -         -          -             STATIC           
...
```

# Other links

## [Debugging with Istio](https://www.istioworkshop.io/12-debugging/01-istioctl-debug-command/)
