mode: job Reference

Full chart values, storage configuration, and RBAC for mode: job


This page is the complete reference for mode: job.

The Job image

The image bundles a pinned version of troubleshoot's support-bundle binary as well as aws-cli and gcloud for bundle upload. For more information, go to Support bundle storage and retrieval.

It also bundles kubectl and curl, which the metrics collector uses to find your router pods and scrape them every time the Job runs. For more information, see Collecting Metrics → mode: job.

Chart values

Chart values common to both modes

Every one of these values applies regardless of which mode you run.

ValueRequiredDefaultApplies toPurpose
namespaceYes—Official chart, raw manifest / customYour router's namespace. Rendered into every collector that accepts it, including clusterResources.namespaces as a single-element list. Never rely on the kubectl context's default namespace; it isn't used.
modeYes—Official chart, raw manifest / customSelects how collection runs: local runs on your own machine using your own kubectl credentials and job runs in-cluster as a Kubernetes Job using a ServiceAccount the chart creates.
selectorRaw-manifest tier onlyThe official chart's app.kubernetes.io/name=router labelRaw manifests / customPod label selector, for example, app=my-router. Also selects which pods the metrics collector scrapes, found each time collection runs. Set it on the official chart too if your router's app.kubernetes.io/name label isn't router, for example with router.nameOverride. For more information, see Collecting Metrics → Which pods get scraped.
configMapNameRaw-manifest tier onlyLabel-based targeting, no name neededRaw manifests / customName of the ConfigMap holding your router's rendered config
metricsPortNo9090Official chart, raw manifest / customPort your router's metrics endpoint is reachable on, if not 9090. For more information, go to Collecting Metrics.
logs.maxAgeNoUnset—no age capOfficial chart, raw manifest / customMaps to the logs collector's limits.maxAge. How far back to reach, for example, 2h
logs.maxLinesNo10000 (troubleshoot.sh default)Official chart, raw manifest / customMaps to limits.maxLines. Lines kept, newest per container

Go to Local Mode / Job Mode to determine what each mode needs beyond this shared set.

Chart values specific to mode: job

ValueRequiredDefaultPurpose
job.image.repositoryNoapollograph/router-diagnosticsThe Job's container image, published by Apollo. Override only to pin an older version or point at a private mirror.
job.image.tagNoMatches this chart's release versionThe troubleshoot.sh version it bundles is a separate, independently-pinned detail, recorded in a collected bundle's meta.json. Override only to pin an older release or point at a private mirror.
job.image.pullPolicyNoIfNotPresentStandard Kubernetes image pull policy
job.collectNodeMetricsNotrueSet false to skip creating the cluster-scoped ClusterRole/ClusterRoleBinding and decline the kubelet read up front. Without them the Job can't read container memory and CPU usage from the kubelet or collect node pressure conditions, and the bundle says the kubelet read was declined. Everything else is still collected. For more information, see Data Collected → Memory and CPU information collected.
job.podAnnotationsNo{sidecar.istio.io/inject: "false", linkerd.io/inject: disabled}Annotations merged onto the Job's pod template—customer values win over these defaults. For more information, go to Job Mode → Service mesh environments
job.podLabelsNo{}Arbitrary labels merged onto the Job's pod template
job.ttlSecondsAfterFinishedNo3600Maps to the Job's ttlSecondsAfterFinished—how long after completion Kubernetes automatically deletes the Job and its pod
job.serviceAccount.createNotrueWhether the chart creates the Job's ServiceAccount (router-diagnostics-job) and the Role, RoleBinding, ClusterRole and ClusterRoleBinding that go with it. Set false to run under your own ServiceAccount, named in job.serviceAccount.name. For more information, see Run the support tool under your own ServiceAccount.
job.serviceAccount.nameOnly when job.serviceAccount.create is falseUnsetName of an existing ServiceAccount in the release namespace for the Job's pod to run as. The install fails if create is false and the name is unset, or if create is true and the name is set.
job.serviceAccount.annotationsOnly if you enable IRSA/Workload Identity{}Merged onto the ServiceAccount the chart creates. Can't be combined with job.serviceAccount.create: false; put the annotations on your own ServiceAccount instead. Carries eks.amazonaws.com/role-arn (IRSA) or iam.gke.io/gcp-service-account (Workload Identity), whichever matches job.storage.provider. For more information, see Support bundle storage and retrieval → Credentials.

Bundle Retrieval

Storage is required for mode: job because there is no way to get the bundle out otherwise. The chart fails the install at render time if mode: job is set and job.storage.provider is left unset. Go to Support bundle storage and retrieval for the full job.storage reference, worked examples per provider, and how to retrieve the bundle from each destination.

Kubernetes RBAC

Installing mode: job requires creating a Job, a namespace-scoped ServiceAccount, a Role/RoleBinding, and—because container memory/CPU come from the kubelet—cluster-scoped RBAC. Creating cluster-scoped RBAC is a broader capability than installing the router needs, so whoever installs the chart in mode: job needs more access than someone running mode: local with their own kubectl credentials. To avoid that, you can run the Job under your own ServiceAccount, in which case installing the chart needs no RBAC permissions.

By default, collection runs as a namespace-scoped ServiceAccount that the chart creates. All namespace-scoped access is read-only (get/list) and covers everything clusterResources/logs/configMap touch:

  • Core: configmaps, endpoints, events, limitranges, persistentvolumeclaims, pods, pods/log, resourcequotas, serviceaccounts, services

  • apps: daemonsets, deployments, replicasets, statefulsets

  • batch: cronjobs, jobs

  • rbac.authorization.k8s.io: roles, rolebindings

  • policy: poddisruptionbudgets

  • networking.k8s.io: ingresses, networkpolicies

  • discovery.k8s.io: endpointslices

  • coordination.k8s.io: leases

The same get/list on pods is what the metrics collector uses to find your router pods each time the Job runs. It reaches each pod's metrics port directly from inside the cluster, so no pods/portforward grant is needed.

Two cluster-scoped grants apply for the kubelet stats read:

  • nodes (list), which surfaces node pressure conditions

  • nodes/stats (get), which surfaces container memory and CPU usage

What each grant permits is listed in Data Collected → Kubelet stats RBAC.

pods/exec is not required: nothing this tool does runs inside your router container.

Run the support tool under your own ServiceAccount

If you'd rather own the permissions yourself, install with job.serviceAccount.create: false and job.serviceAccount.name set to a ServiceAccount that already exists in the namespace you install the chart into:

Bash
1helm install router-diagnostics oci://registry-1.docker.io/apollograph/router-diagnostics-chart \
2  --namespace production \
3  --set namespace=production \
4  --set mode=job \
5  --set job.serviceAccount.create=false \
6  --set job.serviceAccount.name=my-router-diagnostics \
7  --set job.storage.provider=s3 \
8  --set job.storage.bucket=my-router-diagnostics \
9  --set job.storage.s3.region=us-east-1

The chart then creates no ServiceAccount, Role, RoleBinding, ClusterRole or ClusterRoleBinding, so installing it needs only permission to create the Job and the spec ConfigMap. The Job's pod runs as the ServiceAccount you named, and that ServiceAccount must hold the grants the chart would otherwise create. A grant it lacks doesn't fail collection, the bundle reports what couldn't be read.

  • Namespaced: a Role in the router's namespace (the namespace value), bound to your ServiceAccount, with get and list on the resources listed above.

  • Cluster-scoped: a ClusterRole and ClusterRoleBinding with nodes (list) and nodes/stats (get), described in Data Collected → Kubelet stats RBAC. Skip these if you set job.collectNodeMetrics: false.

YAML
1# Namespaced: a Role in the router's namespace, bound to your ServiceAccount
2rules:
3  - apiGroups: [""]
4    resources:
5      - configmaps
6      - endpoints
7      - events
8      - limitranges
9      - persistentvolumeclaims
10      - pods
11      - pods/log
12      - resourcequotas
13      - serviceaccounts
14      - services
15    verbs: ["get", "list"]
16  - apiGroups: ["apps"]
17    resources: ["daemonsets", "deployments", "replicasets", "statefulsets"]
18    verbs: ["get", "list"]
19  - apiGroups: ["batch"]
20    resources: ["cronjobs", "jobs"]
21    verbs: ["get", "list"]
22  - apiGroups: ["rbac.authorization.k8s.io"]
23    resources: ["rolebindings", "roles"]
24    verbs: ["get", "list"]
25  - apiGroups: ["policy"]
26    resources: ["poddisruptionbudgets"]
27    verbs: ["get", "list"]
28  - apiGroups: ["networking.k8s.io"]
29    resources: ["ingresses", "networkpolicies"]
30    verbs: ["get", "list"]
31  - apiGroups: ["discovery.k8s.io"]
32    resources: ["endpointslices"]
33    verbs: ["get", "list"]
34  - apiGroups: ["coordination.k8s.io"]
35    resources: ["leases"]
36    verbs: ["get", "list"]
37---
38# Cluster-scoped: a ClusterRole, bound to your ServiceAccount with a ClusterRoleBinding.
39# Omit it when job.collectNodeMetrics is false.
40rules:
41  - apiGroups: [""]
42    resources: ["nodes"]
43    verbs: ["list"]
44  - apiGroups: [""]
45    resources: ["nodes/stats"]
46    verbs: ["get"]

clusterResources also attempts several cluster-scoped resource types beyond what's listed above (nodes, storage classes, CustomResourceDefinitions, webhook configurations, and more). A <resource>-errors.json file lands alongside its corresponding <resource>.json containing the Kubernetes API error. cluster-resources/auth-cani-list/ is troubleshoot.sh's own self-reported record of every verb/resource/apiGroup combination the collecting identity had at collection time.

Kubelet stats and missing data

The Job reads container memory and CPU usage from each router node's kubelet. If any of it can't be collected, the Job writes a line for each case, saying why, to host-collectors/run-host/router-resource-usage.txt, instead of leaving the section silently empty. The file is empty when everything was collected. A node the Job can't read loses only the usage figures: cluster-resources still provides limits and requests, OOM kills, and restart counts.

Line writtenWhat it means
kubelet stats not collected: declined at install (job.collectNodeMetrics is false)You installed with job.collectNodeMetrics: false. This isn't a failure.
kubelet stats not collected: no pods matched selector <selector> in namespace <namespace>The selector or namespace doesn't match any router pod.
kubelet stats not collected: could not list pods matching selector <selector> in namespace <namespace>The ServiceAccount can't get/list pods, or the API server was unreachable. The details are in host-collectors/run-host/router-resource-usage-stderr.txt.
kubelet stats not collected for pod <pod-name>: pod has no node yetThe pod is Pending, so there's no node to ask.
kubelet stats not collected for node <node-name>: could not reach the kubelet at <node-ip>:10250The network path from the Job's pod to the node's kubelet is blocked, for example by a NetworkPolicy or a firewall. This isn't an RBAC problem.
kubelet stats not collected for node <node-name>: kubelet returned HTTP <code>The kubelet refused the request. Confirm the ServiceAccount has the nodes/stats grant.
kubelet stats not collected for node <node-name>: the kubelet's response could not be readThe kubelet answered 200 with something that isn't its stats.
kubelet stats not collected for pod <pod-name>: not in node <node-name>'s statsThe pod was replaced or deleted between listing it and reading its node.

Cluster footprint

Spec ConfigMap, Helm release metadata, and the ServiceAccount/Role/RoleBinding stay behind after installing. With job.serviceAccount.create: false, the chart creates no RBAC, so only the spec ConfigMap and release metadata remain, and your own ServiceAccount and its grants are left untouched. Each Job is transient on its own: ttlSecondsAfterFinished (default 3600) means Kubernetes automatically deletes that Job and its pod an hour after it completes. Collecting more than once creates a new, uniquely-named Job per run (router-diagnostics-job-<n>, see Collecting again) rather than replacing the previous one, and helm uninstall doesn't delete any Job early — each one only ever cleans up via its own TTL. To remove everything else, including any RBAC the chart created:

Bash
1helm uninstall router-diagnostics --namespace <namespace>

Because helm uninstall leaves not-yet-expired Jobs in place, and a fresh helm install restarts Job naming from router-diagnostics-job-1, reinstalling into the same namespace within the TTL window can collide with a leftover Job of the same name. For a workaround, see Collecting again → Cleaning up.

Bundle output shape

Extracting support-bundle-<timestamp>.tar.gz yields one top-level directory. version.yaml, analysis.json, cluster-info/, and execution-data/summary.txt always land in it, written unconditionally by the engine itself regardless of what the spec declares; meta.json is ours. execution-data/summary.txt is worth a look if something seems off — it's a human-readable per-collector success/failure/timing report.

Worked example, for a mode: job install in the production namespace:

Text
1support-bundle-2026-08-11T14_23_00/
2├── version.yaml
3├── analysis.json
4├── meta.json
5├── cluster-info/
6│   └── cluster_version.json
7├── execution-data/
8│   └── summary.txt
9├── host-collectors/
10│   └── run-host/
11│       ├── router-metrics-info.json                               # fully masked — see Collecting Metrics
12│       ├── router-metrics.txt                                     # the collector's own output, including why any metrics are missing
13│       ├── router-metrics-stderr.txt                              # the script's stderr, only present if the script writes to stderr
14│       ├── router-metrics/
15│       │   └── pods/
16│       │       ├── <pod-name>.txt
17│       │       └── <pod-2-name>.txt
18│       ├── router-resource-usage-info.json                        # fully masked
19│       ├── router-resource-usage.txt                              # empty when everything was collected, otherwise why kubelet stats are missing
20│       ├── router-resource-usage-stderr.txt                       # the script's stderr, only present if the script writes to stderr
21│       └── router-resource-usage/
22│           └── nodes/
23│               └── <node-name>.json
24├── router-logs/
25│   └── <router-pod-name>/
26│       └── router.log -> ../../cluster-resources/pods/logs/production/<router-pod-name>/router.log
27├── cluster-resources/
28│   ├── pods/
29│   │   ├── production.json
30│   │   └── logs/production/<router-pod-name>/router.log         # the real file behind the symlink above
31│   ├── configmaps/production.json
32│   ├── events/production.json
33│   ├── nodes.json
34│   └── ...
35└── configmaps/
36    └── production/
37        └── <configmap-name>.json
  • host-collectors/run-host/router-metrics/pods/ — one <pod-name>.txt per router pod matching selector that responded, holding that pod's Prometheus metrics. See Collecting Metrics → Where metrics land in the bundle for details.

  • host-collectors/run-host/router-metrics.txt — the metrics collector's own output, separate from the per-pod files. It has one line for each pod that couldn't be scraped, or for why no pod could be. See Collecting Metrics → If metrics are missing for what each line means.

  • host-collectors/run-host/router-metrics-stderr.txt — the metrics collector's stderr, such as the error kubectl returned when it couldn't list pods. It is only present if the script writes to stderr. See Collecting Metrics → If metrics are missing.

  • host-collectors/run-host/router-metrics-info.json — a diagnostic sidecar troubleshoot.sh writes for the metrics collector. Its contents are fully masked.

  • The file under configmaps/production/ is named after the matched ConfigMap's own Kubernetes name, on the official chart these happen to coincide, since that chart names the ConfigMap after the release, but a raw-manifest or custom deployment has no Helm release of the router at all. If more than one ConfigMap matched the label selector in production, configmaps/production/ becomes multiple files, one per ConfigMap:

    Text
    1configmaps/
    2└── production/
    3    ├── <configmap-1-name>.json
    4    └── <configmap-2-name>.json
  • host-collectors/run-host/router-resource-usage/nodes/<node-name>.json — one file for each node that runs a router pod, holding that node's figures and the kubelet's per-container memory and CPU usage for the router's pods only. No other workload's pods are included. The files contain pod names, namespaces, and container CPU and memory usage, but no logs, configuration, or secrets. If a node has no file, it wasn't read, and host-collectors/run-host/router-resource-usage.txt says why. See Kubelet stats and missing data.

  • cluster-resources/configmaps/production.json is clusterResources's full, unfiltered sweep of every ConfigMap in the namespace. configmaps/production/<configmap-name>.json is the dedicated configMap collector's own output.

note
keyExists: false in configmaps/<namespace>/<configmap-name>.json doesn't mean the config is missing. This is a troubleshoot.sh quirk: the configMap collector's keyExists field is only ever set true when the collector is configured with a specific key to look for. Our spec uses includeAllData: true with no key set, so keyExists is always false in this output, regardless of whether the config was actually captured. Check for the config data in that same file, rather than reading keyExists as a success/failure signal.