2026-06-05T01:57:47.027320303Z [disk usage checker] Started
2026-06-05T01:57:47.031354966Z [disk usage checker] Volume usage percentage: current = 30 ; allowed = 70
2026-06-05T01:57:47.363598159Z Gathering data for ns/openshift-cluster-version...
2026-06-05T01:57:47.504431518Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T01:57:47.602927494Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T01:57:47.692653077Z Gathering data for ns/default...
2026-06-05T01:57:48.009813800Z Gathering data for ns/openshift...
2026-06-05T01:57:48.423511377Z Gathering data for ns/kube-system...
2026-06-05T01:57:49.090348667Z Gathering data for ns/openshift-etcd...
2026-06-05T01:57:49.413382941Z Wrote inspect data to must-gather.
2026-06-05T01:57:49.413416090Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T01:57:49.413416090Z     namespaces "assisted-installer" not found
2026-06-05T01:57:49.702716838Z Waiting on subprocesses to finish execution.
2026-06-05T01:57:49.709579024Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-05T01:57:49.714544587Z Executing Istio gather script
2026-06-05T01:57:49.718349012Z INFO: Gathering HAProxy config files
2026-06-05T01:57:49.719472432Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-05T01:57:49.720155512Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-05T01:57:49.809515610Z INFO: Collecting host service logs for crio
2026-06-05T01:57:49.809914584Z INFO: Collecting host service logs for kubelet
2026-06-05T01:57:49.810298020Z INFO: Collecting host service logs for rpm-ostreed
2026-06-05T01:57:49.810633353Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-05T01:57:49.810966276Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-05T01:57:49.811312021Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-05T01:57:49.811653305Z INFO: Collecting host service logs for NetworkManager
2026-06-05T01:57:49.811986349Z INFO: Collecting host service logs for openvswitch
2026-06-05T01:57:49.812318747Z INFO: Collecting host service logs for ovs-configuration
2026-06-05T01:57:49.812652870Z INFO: Collecting host service logs for ovsdb-server
2026-06-05T01:57:49.812988491Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-05T01:57:49.813400232Z INFO: Waiting for worker host service log collection to complete ...
2026-06-05T01:57:49.821626897Z INFO: Waiting for node performance related collection to complete ...
2026-06-05T01:57:50.134926363Z No resources found in openshift-etcd namespace.
2026-06-05T01:57:50.137603816Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-05T01:57:50.303803660Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-05T01:57:50.318927198Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-05T01:57:50.319024908Z INFO: on-disk MachineConfig config collection complete.
2026-06-05T01:57:50.614007675Z No resources found
2026-06-05T01:57:50.766727097Z INFO: Collecting Insights Archives from insights-operator-896549976-mvxhz
2026-06-05T01:57:50.767557413Z insights-runtime-extractor-857cc
2026-06-05T01:57:50.767588838Z insights-runtime-extractor-pp456
2026-06-05T01:57:50.767610926Z insights-runtime-extractor-qk9jz
2026-06-05T01:57:50.814082765Z INFO: "metallb-operator" not detected. Skipping.
2026-06-05T01:57:50.850247350Z Wrote inspect data to must-gather.
2026-06-05T01:57:51.025137519Z error: the server doesn't have a resource type "performanceprofile"
2026-06-05T01:57:51.180836989Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-05T01:57:51.233588307Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-05T01:57:51.324985229Z INFO: Worker host service log collection to complete.
2026-06-05T01:57:51.390873212Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-05T01:57:51.390873212Z See 'oc rsync -h' for help and examples
2026-06-05T01:57:51.480863086Z Wrote inspect data to must-gather.
2026-06-05T01:57:51.597993854Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-05T01:57:51.730328382Z Wrote inspect data to must-gather.
2026-06-05T01:57:51.797959076Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-05T01:57:51.898649340Z Wrote inspect data to must-gather.
2026-06-05T01:57:52.011530858Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T01:57:52.020935123Z ERROR: No running kube-apiserver pods found
2026-06-05T01:57:52.073029168Z Gathering data for ns/openshift-monitoring...
2026-06-05T01:57:52.097783845Z [disk usage checker] Volume usage percentage: current = 30 ; allowed = 70
2026-06-05T01:57:52.104805275Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-05T01:57:52.238797077Z No resources found
2026-06-05T01:57:52.255404030Z Wrote inspect data to must-gather.
2026-06-05T01:57:52.284330905Z Wrote inspect data to must-gather.
2026-06-05T01:57:52.292973145Z error: the server doesn't have a resource type "machineconfigs"
2026-06-05T01:57:52.329603039Z Wrote inspect data to must-gather.
2026-06-05T01:57:52.350547990Z INFO: 'previous-logs' folder not found on ip-10-0-131-122.ec2.internal, skipping...
2026-06-05T01:57:52.423431994Z tar: Removing leading `/' from member names
2026-06-05T01:57:52.432325071Z INFO: HAProxy config collection complete.
2026-06-05T01:57:52.490363323Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-05T01:57:52.616013864Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-05T01:57:52.628404693Z INFO: Getting rules from prometheus-k8s-0
2026-06-05T01:57:52.638372258Z Wrote inspect data to must-gather.
2026-06-05T01:57:52.651611102Z INFO: 'previous-logs' folder not found on ip-10-0-138-90.ec2.internal, skipping...
2026-06-05T01:57:52.758417195Z Wrote inspect data to must-gather.
2026-06-05T01:57:52.907800145Z Wrote inspect data to must-gather.
2026-06-05T01:57:52.908018051Z INFO: 'previous-logs' folder not found on ip-10-0-142-29.ec2.internal, skipping...
2026-06-05T01:57:52.908104032Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-05T01:57:52.908154369Z INFO: Machine Config Daemon termination log collection complete.
2026-06-05T01:57:52.931359412Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-05T01:57:53.038731751Z INFO: Getting status/config from prometheus-k8s-0
2026-06-05T01:57:53.148606463Z Wrote inspect data to must-gather.
2026-06-05T01:57:53.171300387Z Wrote inspect data to must-gather.
2026-06-05T01:57:53.353574948Z Wrote inspect data to must-gather.
2026-06-05T01:57:53.439533870Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-05T01:57:53.442058967Z INFO: INTERCONNECT MODE
2026-06-05T01:57:53.442058967Z INFO: Gathering ovn-kubernetes DBs
2026-06-05T01:57:53.571625504Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-05T01:57:53.591331142Z Wrote inspect data to must-gather.
2026-06-05T01:57:53.629963528Z INFO: Gathering OVN_Northbound from ovnkube-node-jw7fd...
2026-06-05T01:57:53.630229616Z INFO: Gathering OVN_Northbound from ovnkube-node-rjgz8...
2026-06-05T01:57:53.630484506Z INFO: Gathering OVN_Northbound from ovnkube-node-wk8sj...
2026-06-05T01:57:53.630915311Z INFO: Gathering OVN_Southbound from ovnkube-node-jw7fd...
2026-06-05T01:57:53.631215017Z INFO: Gathering OVN_Southbound from ovnkube-node-rjgz8...
2026-06-05T01:57:53.631523394Z INFO: Gathering OVN_Southbound from ovnkube-node-wk8sj...
2026-06-05T01:57:53.682516556Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T01:57:53.777317361Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-05T01:57:53.921296644Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:53.969257105Z Wrote inspect data to must-gather.
2026-06-05T01:57:53.970424167Z tar: Removing leading `/' from member names
2026-06-05T01:57:53.979316643Z tar: Removing leading `/' from member names
2026-06-05T01:57:54.056502240Z tar: Removing leading `/' from member names
2026-06-05T01:57:54.056547786Z tar: Removing leading `/' from member names
2026-06-05T01:57:54.067977007Z tar: Removing leading `/' from member names
2026-06-05T01:57:54.078156533Z tar: Removing leading `/' from member names
2026-06-05T01:57:54.151876168Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-05T01:57:54.197294942Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:54.201476474Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-05T01:57:54.224575743Z Wrote inspect data to must-gather.
2026-06-05T01:57:54.437774414Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-05T01:57:54.461532612Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:54.486843769Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T01:57:54.577345815Z Wrote inspect data to must-gather.
2026-06-05T01:57:54.581346180Z INFO: Waiting for network log collection to complete ...
2026-06-05T01:57:54.581374689Z INFO: Waiting for ovnk database copies to complete ...
2026-06-05T01:57:54.581752475Z INFO: Copying ovnk databases complete.
2026-06-05T01:57:54.583527350Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-06-05T01:57:54.586292841Z ovnk_database_store/
2026-06-05T01:57:54.586316426Z ovnk_database_store/ovnkube-node-jw7fd_nbdb
2026-06-05T01:57:54.596234618Z ovnk_database_store/ovnkube-node-rjgz8_sbdb
2026-06-05T01:57:54.614290870Z ovnk_database_store/ovnkube-node-wk8sj_nbdb
2026-06-05T01:57:54.620748036Z ovnk_database_store/ovnkube-node-wk8sj_sbdb
2026-06-05T01:57:54.642808918Z ovnk_database_store/ovnkube-node-rjgz8_nbdb
2026-06-05T01:57:54.650184319Z ovnk_database_store/ovnkube-node-jw7fd_sbdb
2026-06-05T01:57:54.670761831Z INFO: Network log collection complete.
2026-06-05T01:57:54.696556869Z Gathering data for ns/openshift-network-console...
2026-06-05T01:57:54.701199240Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:54.704320047Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-05T01:57:54.836666249Z INFO: Getting status from alertmanager-main-0
2026-06-05T01:57:54.873727288Z Wrote inspect data to must-gather.
2026-06-05T01:57:54.913574576Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:55.151070422Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:55.153848598Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-05T01:57:55.155119154Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-05T01:57:55.157762071Z INFO: Fallback to identify the container image from release info
2026-06-05T01:57:55.176979690Z Gathering data for ns/openshift-console-operator...
2026-06-05T01:57:55.358015193Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:55.541309425Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:55.544211587Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-05T01:57:55.559292640Z Gathering data for ns/openshift-console...
2026-06-05T01:57:55.705753496Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:55.860465439Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-05T01:57:55.903524123Z Wrote inspect data to /must-gather/istio.
2026-06-05T01:57:55.992937161Z daemonset.apps/perf-node-gather-daemonset created
2026-06-05T01:57:56.112153579Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-05T01:57:56.127701719Z Waiting for performance profile collector pods to become ready: 1
2026-06-05T01:57:56.621509034Z Done executing Istio gather script
2026-06-05T01:57:56.646454470Z Gathering data for ns/openshift-dns-operator...
2026-06-05T01:57:56.909538170Z Gathering data for ns/openshift-dns...
2026-06-05T01:57:57.114176845Z [disk usage checker] Volume usage percentage: current = 30 ; allowed = 70
2026-06-05T01:57:57.241923675Z Waiting for performance profile collector pods to become ready: 2
2026-06-05T01:57:57.469796519Z Gathering data for ns/openshift-image-registry...
2026-06-05T01:57:57.905530985Z Gathering data for ns/openshift-ingress-operator...
2026-06-05T01:57:58.209518560Z Gathering data for ns/openshift-ingress...
2026-06-05T01:57:58.361472366Z Waiting for performance profile collector pods to become ready: 3
2026-06-05T01:57:58.540235130Z Gathering data for ns/openshift-ingress-canary...
2026-06-05T01:57:58.936952098Z Gathering data for ns/openshift-insights...
2026-06-05T01:57:59.472094192Z Waiting for performance profile collector pods to become ready: 4
2026-06-05T01:58:00.585885049Z Waiting for performance profile collector pods to become ready: 5
2026-06-05T01:58:00.777521104Z Gathering data for ns/openshift-jobset-operator...
2026-06-05T01:58:01.543981912Z Gathering data for ns/openshift-config...
2026-06-05T01:58:01.701109666Z Waiting for performance profile collector pods to become ready: 6
2026-06-05T01:58:01.801405289Z Gathering data for ns/openshift-config-managed...
2026-06-05T01:58:02.118989332Z [disk usage checker] Volume usage percentage: current = 30 ; allowed = 70
2026-06-05T01:58:02.194105693Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-05T01:58:02.447513276Z Gathering data for ns/openshift-kube-apiserver...
2026-06-05T01:58:02.707156090Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-05T01:58:02.826639174Z Waiting for performance profile collector pods to become ready: 7
2026-06-05T01:58:02.995177511Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-05T01:58:03.265138937Z Gathering data for ns/openshift-kube-scheduler...
2026-06-05T01:58:03.553402308Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-05T01:58:03.829716190Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-05T01:58:03.945575538Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-05T01:58:04.189030124Z Collecting performance related data for node ip-10-0-131-122.ec2.internal
2026-06-05T01:58:04.193570985Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-05T01:58:04.198597748Z Collecting performance related data for node ip-10-0-142-29.ec2.internal
2026-06-05T01:58:04.206686553Z Collecting performance related data for node ip-10-0-138-90.ec2.internal
2026-06-05T01:58:04.568615485Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-05T01:58:05.192585189Z Gathering data for ns/openshift-multus...
2026-06-05T01:58:06.618431696Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-05T01:58:07.124429989Z [disk usage checker] Volume usage percentage: current = 30 ; allowed = 70
2026-06-05T01:58:07.842307035Z Gathering data for ns/openshift-host-network...
2026-06-05T01:58:08.138922529Z Gathering data for ns/openshift-network-diagnostics...
2026-06-05T01:58:08.674218922Z Gathering data for ns/openshift-network-node-identity...
2026-06-05T01:58:09.125280344Z Gathering data for ns/openshift-network-operator...
2026-06-05T01:58:09.522259780Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-05T01:58:09.797367963Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-05T01:58:10.287730212Z Gathering data for ns/openshift-apiserver-operator...
2026-06-05T01:58:10.703739424Z Gathering data for ns/openshift-apiserver...
2026-06-05T01:58:11.172493891Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-05T01:58:11.437859777Z Gathering data for ns/openshift-controller-manager...
2026-06-05T01:58:11.720160374Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-05T01:58:12.069451117Z Collecting kubelet logs for node ip-10-0-131-122.ec2.internal
2026-06-05T01:58:12.071427349Z Collecting kubelet logs for node ip-10-0-142-29.ec2.internal
2026-06-05T01:58:12.074069307Z Collecting kubelet logs for node ip-10-0-138-90.ec2.internal
2026-06-05T01:58:12.129255392Z [disk usage checker] Volume usage percentage: current = 30 ; allowed = 70
2026-06-05T01:58:12.389947802Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-t6j25 namespace
2026-06-05T01:58:12.396164688Z INFO: Node performance data collection complete.
2026-06-05T01:58:12.435315249Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-05T01:58:12.719427481Z Gathering data for ns/openshift-service-ca-operator...
2026-06-05T01:58:13.017785957Z Gathering data for ns/openshift-service-ca...
2026-06-05T01:58:13.356304286Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-05T01:58:14.204755433Z Wrote inspect data to must-gather.
2026-06-05T01:58:14.204800941Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T01:58:14.204800941Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-05T01:58:14.323397366Z error: the server doesn't have a resource type "clusters"
2026-06-05T01:58:14.419085617Z Caches written to disk