2026-09-16T07:21:32.531310718Z [disk usage checker] Started
2026-09-16T07:21:32.535055296Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T07:21:32.734996284Z Gathering data for ns/openshift-cluster-version...
2026-09-16T07:21:32.891384894Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T07:21:32.993798164Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T07:21:33.077395892Z Gathering data for ns/default...
2026-09-16T07:21:33.398615207Z Gathering data for ns/openshift...
2026-09-16T07:21:33.846432337Z Gathering data for ns/kube-system...
2026-09-16T07:21:34.546984403Z Gathering data for ns/openshift-etcd...
2026-09-16T07:21:34.836517967Z Wrote inspect data to must-gather.
2026-09-16T07:21:34.836554590Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T07:21:34.836554590Z     namespaces "assisted-installer" not found
2026-09-16T07:21:35.046434029Z Waiting on subprocesses to finish execution.
2026-09-16T07:21:35.052569791Z INFO: Gathering HAProxy config files
2026-09-16T07:21:35.055177198Z INFO: Collecting host service logs for crio
2026-09-16T07:21:35.055546610Z INFO: Collecting host service logs for kubelet
2026-09-16T07:21:35.055873535Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T07:21:35.056189234Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T07:21:35.056540739Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T07:21:35.056876563Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T07:21:35.057230497Z INFO: Collecting host service logs for NetworkManager
2026-09-16T07:21:35.057580972Z INFO: Collecting host service logs for openvswitch
2026-09-16T07:21:35.057987090Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T07:21:35.058369964Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T07:21:35.058700879Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T07:21:35.059082185Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T07:21:35.061879889Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T07:21:35.066890397Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T07:21:35.078889941Z Executing Istio gather script
2026-09-16T07:21:35.105030460Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T07:21:35.111005699Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T07:21:35.592316834Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T07:21:35.627627484Z INFO: Collecting Insights Archives from insights-operator-5844b856db-zznkh
2026-09-16T07:21:35.627627484Z insights-runtime-extractor-2rl8v
2026-09-16T07:21:35.627627484Z insights-runtime-extractor-cppd4
2026-09-16T07:21:35.627627484Z insights-runtime-extractor-q557r
2026-09-16T07:21:36.061293202Z No resources found in openshift-etcd namespace.
2026-09-16T07:21:36.097352121Z No resources found
2026-09-16T07:21:36.315471445Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T07:21:36.356271071Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T07:21:36.370259997Z Wrote inspect data to must-gather.
2026-09-16T07:21:36.399664299Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T07:21:36.399664299Z See 'oc rsync -h' for help and examples
2026-09-16T07:21:36.407571358Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T07:21:36.407603679Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T07:21:36.474244741Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T07:21:36.477934730Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T07:21:36.557886241Z Wrote inspect data to must-gather.
2026-09-16T07:21:36.752582283Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T07:21:36.799586839Z Wrote inspect data to must-gather.
2026-09-16T07:21:36.847495632Z INFO: Worker host service log collection to complete.
2026-09-16T07:21:36.972525196Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T07:21:37.053428357Z ERROR: No running kube-apiserver pods found
2026-09-16T07:21:37.057511727Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T07:21:37.253161909Z Wrote inspect data to must-gather.
2026-09-16T07:21:37.288670044Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T07:21:37.305851300Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T07:21:37.364792881Z Wrote inspect data to must-gather.
2026-09-16T07:21:37.394443411Z No resources found
2026-09-16T07:21:37.418176942Z Wrote inspect data to must-gather.
2026-09-16T07:21:37.464988846Z INFO: 'previous-logs' folder not found on ip-10-0-132-34.ec2.internal, skipping...
2026-09-16T07:21:37.522243685Z Wrote inspect data to must-gather.
2026-09-16T07:21:37.544837690Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T07:21:37.561117184Z Gathering data for ns/openshift-monitoring...
2026-09-16T07:21:37.642325671Z tar: Removing leading `/' from member names
2026-09-16T07:21:37.646747706Z INFO: HAProxy config collection complete.
2026-09-16T07:21:37.731936925Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T07:21:37.768808536Z Wrote inspect data to must-gather.
2026-09-16T07:21:37.816481545Z INFO: 'previous-logs' folder not found on ip-10-0-132-37.ec2.internal, skipping...
2026-09-16T07:21:37.857382385Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T07:21:37.930467107Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T07:21:37.943745359Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T07:21:38.087794494Z Wrote inspect data to must-gather.
2026-09-16T07:21:38.134544343Z INFO: 'previous-logs' folder not found on ip-10-0-139-18.ec2.internal, skipping...
2026-09-16T07:21:38.134645211Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T07:21:38.134693236Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T07:21:38.213875408Z Wrote inspect data to must-gather.
2026-09-16T07:21:38.267692811Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T07:21:38.344340372Z Wrote inspect data to must-gather.
2026-09-16T07:21:38.375174860Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T07:21:38.578103136Z Wrote inspect data to must-gather.
2026-09-16T07:21:38.620190574Z INFO: INTERCONNECT MODE
2026-09-16T07:21:38.620247744Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T07:21:38.657332760Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T07:21:38.798423107Z Wrote inspect data to must-gather.
2026-09-16T07:21:38.798682922Z INFO: Gathering OVN_Northbound from ovnkube-node-2pkws...
2026-09-16T07:21:38.799076084Z INFO: Gathering OVN_Northbound from ovnkube-node-9tksk...
2026-09-16T07:21:38.799388934Z INFO: Gathering OVN_Northbound from ovnkube-node-r25l4...
2026-09-16T07:21:38.799717650Z INFO: Gathering OVN_Southbound from ovnkube-node-2pkws...
2026-09-16T07:21:38.800004103Z INFO: Gathering OVN_Southbound from ovnkube-node-9tksk...
2026-09-16T07:21:38.800329321Z INFO: Gathering OVN_Southbound from ovnkube-node-r25l4...
2026-09-16T07:21:38.902518468Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T07:21:38.933360633Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T07:21:39.105862940Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T07:21:39.109453393Z tar: Removing leading `/' from member names
2026-09-16T07:21:39.134941129Z tar: Removing leading `/' from member names
2026-09-16T07:21:39.136901282Z tar: Removing leading `/' from member names
2026-09-16T07:21:39.175760890Z tar: Removing leading `/' from member names
2026-09-16T07:21:39.179782872Z Wrote inspect data to must-gather.
2026-09-16T07:21:39.229907662Z tar: Removing leading `/' from member names
2026-09-16T07:21:39.233688432Z tar: Removing leading `/' from member names
2026-09-16T07:21:39.243569269Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:39.259855558Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T07:21:39.424522341Z Wrote inspect data to must-gather.
2026-09-16T07:21:39.514507680Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:39.517059578Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T07:21:39.517595354Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T07:21:39.664503001Z W0916 07:21:39.664456    1451 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-16T07:21:39.670531285Z Wrote inspect data to must-gather.
2026-09-16T07:21:39.672607862Z INFO: Waiting for network log collection to complete ...
2026-09-16T07:21:39.672638179Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T07:21:39.673007539Z INFO: Copying ovnk databases complete.
2026-09-16T07:21:39.674723893Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-16T07:21:39.677251870Z ovnk_database_store/
2026-09-16T07:21:39.677322848Z ovnk_database_store/ovnkube-node-2pkws_sbdb
2026-09-16T07:21:39.693229747Z ovnk_database_store/ovnkube-node-r25l4_sbdb
2026-09-16T07:21:39.711876253Z ovnk_database_store/ovnkube-node-2pkws_nbdb
2026-09-16T07:21:39.721126381Z ovnk_database_store/ovnkube-node-9tksk_nbdb
2026-09-16T07:21:39.730368490Z ovnk_database_store/ovnkube-node-9tksk_sbdb
2026-09-16T07:21:39.741184627Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:39.753085925Z ovnk_database_store/ovnkube-node-r25l4_nbdb
2026-09-16T07:21:39.766569045Z INFO: Network log collection complete.
2026-09-16T07:21:39.771798580Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T07:21:39.939154461Z INFO: Getting status from alertmanager-main-0
2026-09-16T07:21:39.968294169Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:39.971242922Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T07:21:39.972395754Z Wrote inspect data to must-gather.
2026-09-16T07:21:39.982755467Z Gathering data for ns/openshift-network-console...
2026-09-16T07:21:40.166101665Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:40.240597347Z Wrote inspect data to must-gather.
2026-09-16T07:21:40.371125217Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:40.374490598Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T07:21:40.442038080Z Gathering data for ns/openshift-console-operator...
2026-09-16T07:21:40.517805749Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T07:21:40.520284421Z INFO: Fallback to identify the container image from release info
2026-09-16T07:21:40.600965481Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:40.794298837Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:40.797572125Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T07:21:40.831951199Z Gathering data for ns/openshift-console...
2026-09-16T07:21:40.953848140Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:41.035177717Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T07:21:41.140201640Z Wrote inspect data to /must-gather/istio.
2026-09-16T07:21:41.163392475Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T07:21:41.303651205Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T07:21:41.353657549Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T07:21:41.838685072Z Done executing Istio gather script
2026-09-16T07:21:41.865292837Z Gathering data for ns/openshift-dns-operator...
2026-09-16T07:21:42.159572581Z Gathering data for ns/openshift-dns...
2026-09-16T07:21:42.414515134Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T07:21:42.552244476Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T07:21:42.734821146Z Gathering data for ns/openshift-image-registry...
2026-09-16T07:21:43.174496911Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T07:21:43.526628191Z Gathering data for ns/openshift-ingress...
2026-09-16T07:21:43.538718868Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T07:21:43.828739988Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T07:21:44.181502844Z Gathering data for ns/openshift-insights...
2026-09-16T07:21:44.642715079Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T07:21:45.757805328Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T07:21:45.988747434Z Gathering data for ns/openshift-jobset-operator...
2026-09-16T07:21:46.737844537Z Gathering data for ns/openshift-config...
2026-09-16T07:21:46.870938434Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T07:21:47.014050098Z Gathering data for ns/openshift-config-managed...
2026-09-16T07:21:47.402348015Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T07:21:47.557888828Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T07:21:47.646306307Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T07:21:47.893776334Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T07:21:47.990114876Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T07:21:48.170234576Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T07:21:48.435733419Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T07:21:48.664888575Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T07:21:48.926435049Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T07:21:49.106170287Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T07:21:49.250426301Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T07:21:49.356841654Z Collecting performance related data for node ip-10-0-132-37.ec2.internal
2026-09-16T07:21:49.363429968Z Collecting performance related data for node ip-10-0-139-18.ec2.internal
2026-09-16T07:21:49.370780321Z Collecting performance related data for node ip-10-0-132-34.ec2.internal
2026-09-16T07:21:49.583947992Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T07:21:50.145580775Z Gathering data for ns/openshift-multus...
2026-09-16T07:21:51.566330287Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T07:21:52.563293509Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T07:21:52.784329232Z Gathering data for ns/openshift-host-network...
2026-09-16T07:21:53.116490224Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T07:21:53.628927264Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T07:21:54.052813062Z Gathering data for ns/openshift-network-operator...
2026-09-16T07:21:54.447287508Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T07:21:54.700780481Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T07:21:55.163078200Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T07:21:55.470955490Z Gathering data for ns/openshift-apiserver...
2026-09-16T07:21:55.825654113Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T07:21:56.071147169Z Gathering data for ns/openshift-controller-manager...
2026-09-16T07:21:56.466741584Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T07:21:57.083825388Z Collecting kubelet logs for node ip-10-0-132-37.ec2.internal
2026-09-16T07:21:57.084237937Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T07:21:57.085629005Z Collecting kubelet logs for node ip-10-0-139-18.ec2.internal
2026-09-16T07:21:57.087473999Z Collecting kubelet logs for node ip-10-0-132-34.ec2.internal
2026-09-16T07:21:57.418666583Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T07:21:57.424752963Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4kl9r namespace
2026-09-16T07:21:57.429861126Z INFO: Node performance data collection complete.
2026-09-16T07:21:57.568719439Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-16T07:21:57.729423956Z Gathering data for ns/openshift-service-ca...
2026-09-16T07:21:58.060561115Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T07:21:58.935037568Z Wrote inspect data to must-gather.
2026-09-16T07:21:58.935082381Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T07:21:58.935082381Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T07:21:59.056449820Z error: the server doesn't have a resource type "clusters"
2026-09-16T07:21:59.217285059Z Caches written to disk