2026-07-08T17:03:05.574759429Z [disk usage checker] Started
2026-07-08T17:03:05.577897085Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-08T17:03:05.778317261Z Gathering data for ns/openshift-cluster-version...
2026-07-08T17:03:05.957686972Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-08T17:03:06.127858239Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T17:03:06.233918003Z Gathering data for ns/default...
2026-07-08T17:03:06.565306493Z Gathering data for ns/openshift...
2026-07-08T17:03:06.982672791Z Gathering data for ns/kube-system...
2026-07-08T17:03:07.787113980Z Gathering data for ns/openshift-etcd...
2026-07-08T17:03:08.063023839Z Waiting on subprocesses to finish execution.
2026-07-08T17:03:08.075099724Z INFO: Gathering HAProxy config files
2026-07-08T17:03:08.078930264Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-08T17:03:08.081404405Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-08T17:03:08.086281072Z Executing Istio gather script
2026-07-08T17:03:08.112228488Z INFO: Waiting for node performance related collection to complete ...
2026-07-08T17:03:08.127313474Z Wrote inspect data to must-gather.
2026-07-08T17:03:08.127398668Z error: inspection completed with the errors occurred while gathering data:
2026-07-08T17:03:08.127398668Z     namespaces "assisted-installer" not found
2026-07-08T17:03:08.130661664Z INFO: Collecting host service logs for crio
2026-07-08T17:03:08.131077504Z INFO: Collecting host service logs for kubelet
2026-07-08T17:03:08.131439789Z INFO: Collecting host service logs for rpm-ostreed
2026-07-08T17:03:08.132470417Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-08T17:03:08.132899522Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-08T17:03:08.133563028Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-08T17:03:08.134250487Z INFO: Collecting host service logs for NetworkManager
2026-07-08T17:03:08.134925451Z INFO: Collecting host service logs for openvswitch
2026-07-08T17:03:08.135442744Z INFO: Collecting host service logs for ovs-configuration
2026-07-08T17:03:08.135811837Z INFO: Collecting host service logs for ovsdb-server
2026-07-08T17:03:08.136190670Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-08T17:03:08.136644307Z INFO: Waiting for worker host service log collection to complete ...
2026-07-08T17:03:08.177886058Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-08T17:03:08.329585850Z No resources found in openshift-etcd namespace.
2026-07-08T17:03:08.747980406Z error: the server doesn't have a resource type "performanceprofile"
2026-07-08T17:03:08.955045110Z Wrote inspect data to must-gather.
2026-07-08T17:03:08.968550985Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-08T17:03:08.987046306Z No resources found
2026-07-08T17:03:09.119157939Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-08T17:03:09.119239076Z INFO: on-disk MachineConfig config collection complete.
2026-07-08T17:03:09.270810509Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-08T17:03:09.270810509Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-08T17:03:09.272624559Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-08T17:03:09.329263781Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-w47s9
2026-07-08T17:03:09.329263781Z insights-runtime-extractor-pnthb
2026-07-08T17:03:09.329263781Z insights-runtime-extractor-wjs9j
2026-07-08T17:03:09.329263781Z insights-runtime-extractor-zqtkk
2026-07-08T17:03:09.589940888Z INFO: "metallb-operator" not detected. Skipping.
2026-07-08T17:03:09.658931746Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-08T17:03:09.750611808Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-08T17:03:09.750611808Z See 'oc rsync -h' for help and examples
2026-07-08T17:03:09.953167000Z ERROR: No running kube-apiserver pods found
2026-07-08T17:03:09.983440758Z Wrote inspect data to must-gather.
2026-07-08T17:03:10.044380781Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T17:03:10.049265347Z Wrote inspect data to must-gather.
2026-07-08T17:03:10.140631212Z Wrote inspect data to must-gather.
2026-07-08T17:03:10.219361918Z INFO: Worker host service log collection to complete.
2026-07-08T17:03:10.276176280Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-08T17:03:10.340205164Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-08T17:03:10.391597486Z No resources found
2026-07-08T17:03:10.393477870Z Wrote inspect data to must-gather.
2026-07-08T17:03:10.403253384Z Wrote inspect data to must-gather.
2026-07-08T17:03:10.423035419Z Wrote inspect data to must-gather.
2026-07-08T17:03:10.510588813Z error: the server doesn't have a resource type "machineconfigs"
2026-07-08T17:03:10.531257998Z tar: Removing leading `/' from member names
2026-07-08T17:03:10.535728667Z INFO: HAProxy config collection complete.
2026-07-08T17:03:10.552580701Z INFO: 'previous-logs' folder not found on ip-10-0-129-229.ec2.internal, skipping...
2026-07-08T17:03:10.583693781Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-08T17:03:10.720927393Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-08T17:03:10.734743439Z Gathering data for ns/openshift-monitoring...
2026-07-08T17:03:10.770415426Z Wrote inspect data to must-gather.
2026-07-08T17:03:10.810223154Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-08T17:03:10.860182254Z INFO: 'previous-logs' folder not found on ip-10-0-129-40.ec2.internal, skipping...
2026-07-08T17:03:10.931773497Z INFO: Getting rules from prometheus-k8s-0
2026-07-08T17:03:10.968159543Z Wrote inspect data to must-gather.
2026-07-08T17:03:11.053258104Z Wrote inspect data to must-gather.
2026-07-08T17:03:11.128243440Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-08T17:03:11.135021513Z INFO: 'previous-logs' folder not found on ip-10-0-143-156.ec2.internal, skipping...
2026-07-08T17:03:11.135084139Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-08T17:03:11.135142628Z INFO: Machine Config Daemon termination log collection complete.
2026-07-08T17:03:11.302405711Z Wrote inspect data to must-gather.
2026-07-08T17:03:11.338674426Z INFO: Getting status/config from prometheus-k8s-0
2026-07-08T17:03:11.353737988Z Wrote inspect data to must-gather.
2026-07-08T17:03:11.568465118Z Wrote inspect data to must-gather.
2026-07-08T17:03:11.570718526Z INFO: INTERCONNECT MODE
2026-07-08T17:03:11.570741840Z INFO: Gathering ovn-kubernetes DBs
2026-07-08T17:03:11.735133022Z INFO: Gathering OVN_Northbound from ovnkube-node-8ptd2...
2026-07-08T17:03:11.735388226Z INFO: Gathering OVN_Northbound from ovnkube-node-px95b...
2026-07-08T17:03:11.735641995Z INFO: Gathering OVN_Northbound from ovnkube-node-stsqb...
2026-07-08T17:03:11.735912238Z INFO: Gathering OVN_Southbound from ovnkube-node-8ptd2...
2026-07-08T17:03:11.736202458Z INFO: Gathering OVN_Southbound from ovnkube-node-px95b...
2026-07-08T17:03:11.736496328Z INFO: Gathering OVN_Southbound from ovnkube-node-stsqb...
2026-07-08T17:03:11.772605381Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-08T17:03:11.777603580Z Wrote inspect data to must-gather.
2026-07-08T17:03:11.830995428Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-08T17:03:12.046975195Z tar: Removing leading `/' from member names
2026-07-08T17:03:12.060572910Z tar: Removing leading `/' from member names
2026-07-08T17:03:12.060843489Z tar: Removing leading `/' from member names
2026-07-08T17:03:12.064206803Z tar: Removing leading `/' from member names
2026-07-08T17:03:12.093611605Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:12.100161871Z tar: Removing leading `/' from member names
2026-07-08T17:03:12.120669464Z tar: Removing leading `/' from member names
2026-07-08T17:03:12.146914788Z Wrote inspect data to must-gather.
2026-07-08T17:03:12.159182828Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-08T17:03:12.328848311Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:12.332246059Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-08T17:03:12.379623327Z Wrote inspect data to must-gather.
2026-07-08T17:03:12.416557533Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-08T17:03:12.522378530Z INFO: Waiting for network log collection to complete ...
2026-07-08T17:03:12.522506130Z INFO: Waiting for ovnk database copies to complete ...
2026-07-08T17:03:12.523020694Z INFO: Copying ovnk databases complete.
2026-07-08T17:03:12.525450498Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-07-08T17:03:12.530487010Z ovnk_database_store/
2026-07-08T17:03:12.530537380Z ovnk_database_store/ovnkube-node-px95b_sbdb
2026-07-08T17:03:12.551645415Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:12.551756850Z ovnk_database_store/ovnkube-node-8ptd2_sbdb
2026-07-08T17:03:12.567643844Z ovnk_database_store/ovnkube-node-stsqb_sbdb
2026-07-08T17:03:12.585247123Z ovnk_database_store/ovnkube-node-8ptd2_nbdb
2026-07-08T17:03:12.588407005Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-08T17:03:12.591347226Z ovnk_database_store/ovnkube-node-px95b_nbdb
2026-07-08T17:03:12.597867091Z ovnk_database_store/ovnkube-node-stsqb_nbdb
2026-07-08T17:03:12.608553820Z INFO: Network log collection complete.
2026-07-08T17:03:12.636483668Z Wrote inspect data to must-gather.
2026-07-08T17:03:12.696474959Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-08T17:03:12.768303296Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:12.771756388Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-08T17:03:12.853199342Z Wrote inspect data to must-gather.
2026-07-08T17:03:12.970280766Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:13.098902723Z INFO: Getting status from alertmanager-main-0
2026-07-08T17:03:13.115740192Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-08T17:03:13.118207962Z INFO: Fallback to identify the container image from release info
2026-07-08T17:03:13.182772221Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:13.185147415Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-08T17:03:13.241045433Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-08T17:03:13.398531464Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:13.407490314Z Gathering data for ns/openshift-network-console...
2026-07-08T17:03:13.581350143Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:13.583678629Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-08T17:03:13.730760476Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-08T17:03:13.756533532Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:13.848596376Z Gathering data for ns/openshift-console-operator...
2026-07-08T17:03:13.865999822Z daemonset.apps/perf-node-gather-daemonset created
2026-07-08T17:03:13.989695240Z Wrote inspect data to /must-gather/istio.
2026-07-08T17:03:14.004678907Z Waiting for performance profile collector pods to become ready: 1
2026-07-08T17:03:14.271165246Z Gathering data for ns/openshift-console...
2026-07-08T17:03:14.621853336Z Done executing Istio gather script
2026-07-08T17:03:14.891037168Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-08T17:03:15.113806947Z Waiting for performance profile collector pods to become ready: 2
2026-07-08T17:03:15.365401298Z Gathering data for ns/openshift-dns-operator...
2026-07-08T17:03:15.594144852Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-08T17:03:15.638748007Z Gathering data for ns/openshift-dns...
2026-07-08T17:03:16.232752663Z Waiting for performance profile collector pods to become ready: 3
2026-07-08T17:03:16.278893779Z Gathering data for ns/openshift-image-registry...
2026-07-08T17:03:16.753528886Z Gathering data for ns/openshift-ingress-operator...
2026-07-08T17:03:17.138607920Z Gathering data for ns/openshift-ingress...
2026-07-08T17:03:17.330133732Z Waiting for performance profile collector pods to become ready: 4
2026-07-08T17:03:17.477468068Z Gathering data for ns/openshift-ingress-canary...
2026-07-08T17:03:17.879655552Z Gathering data for ns/openshift-insights...
2026-07-08T17:03:18.444590429Z Waiting for performance profile collector pods to become ready: 5
2026-07-08T17:03:19.547453501Z Waiting for performance profile collector pods to become ready: 6
2026-07-08T17:03:19.822538403Z Gathering data for ns/openshift-jobset-operator...
2026-07-08T17:03:20.598950491Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-08T17:03:20.624359877Z Gathering data for ns/openshift-config...
2026-07-08T17:03:20.660147141Z Waiting for performance profile collector pods to become ready: 7
2026-07-08T17:03:20.927690309Z Gathering data for ns/openshift-config-managed...
2026-07-08T17:03:21.374957615Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-08T17:03:21.646365900Z Gathering data for ns/openshift-kube-apiserver...
2026-07-08T17:03:21.775708265Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-08T17:03:21.954650304Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-08T17:03:22.032042966Z Collecting performance related data for node ip-10-0-129-40.ec2.internal
2026-07-08T17:03:22.038718678Z Collecting performance related data for node ip-10-0-143-156.ec2.internal
2026-07-08T17:03:22.045598247Z Collecting performance related data for node ip-10-0-129-229.ec2.internal
2026-07-08T17:03:22.290027189Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-08T17:03:22.682669561Z Gathering data for ns/openshift-kube-scheduler...
2026-07-08T17:03:23.023413339Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-08T17:03:23.520724680Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-08T17:03:24.010818706Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-08T17:03:24.339122967Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-08T17:03:24.898888823Z Gathering data for ns/openshift-multus...
2026-07-08T17:03:25.603407882Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-08T17:03:26.467159412Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-08T17:03:27.847212306Z Gathering data for ns/openshift-host-network...
2026-07-08T17:03:28.177281496Z Gathering data for ns/openshift-network-diagnostics...
2026-07-08T17:03:28.736191638Z Gathering data for ns/openshift-network-node-identity...
2026-07-08T17:03:29.170106509Z Gathering data for ns/openshift-network-operator...
2026-07-08T17:03:29.613135358Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-08T17:03:29.904989377Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-08T17:03:29.913343358Z Collecting kubelet logs for node ip-10-0-129-40.ec2.internal
2026-07-08T17:03:29.914984793Z Collecting kubelet logs for node ip-10-0-143-156.ec2.internal
2026-07-08T17:03:29.916672441Z Collecting kubelet logs for node ip-10-0-129-229.ec2.internal
2026-07-08T17:03:30.237190029Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-gsbb8 namespace
2026-07-08T17:03:30.242044757Z INFO: Node performance data collection complete.
2026-07-08T17:03:30.482661546Z Gathering data for ns/openshift-apiserver-operator...
2026-07-08T17:03:30.607719120Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-08T17:03:30.841537349Z Gathering data for ns/openshift-apiserver...
2026-07-08T17:03:31.244320228Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-08T17:03:31.515397910Z Gathering data for ns/openshift-controller-manager...
2026-07-08T17:03:31.864510669Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-08T17:03:32.503843264Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-08T17:03:32.817288214Z Gathering data for ns/openshift-service-ca-operator...
2026-07-08T17:03:33.130991608Z Gathering data for ns/openshift-service-ca...
2026-07-08T17:03:33.543011824Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-08T17:03:34.765197003Z Wrote inspect data to must-gather.
2026-07-08T17:03:34.765239964Z error: inspection completed with the errors occurred while gathering data:
2026-07-08T17:03:34.765239964Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-08T17:03:34.885233339Z error: the server doesn't have a resource type "clusters"
2026-07-08T17:03:35.018961530Z Caches written to disk