2026-04-22T19:35:01.241063651Z [disk usage checker] Started
2026-04-22T19:35:01.274786732Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:35:01.455826417Z Gathering data for ns/openshift-cluster-version...
2026-04-22T19:35:01.597754876Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T19:35:01.677531101Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:35:01.760272877Z Gathering data for ns/default...
2026-04-22T19:35:02.081674429Z Gathering data for ns/openshift...
2026-04-22T19:35:02.496295278Z Gathering data for ns/kube-system...
2026-04-22T19:35:03.183956346Z Gathering data for ns/openshift-etcd...
2026-04-22T19:35:03.477862980Z Wrote inspect data to must-gather.
2026-04-22T19:35:03.477924092Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T19:35:03.477924092Z     namespaces "assisted-installer" not found
2026-04-22T19:35:03.748781952Z Waiting on subprocesses to finish execution.
2026-04-22T19:35:03.759024109Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T19:35:03.762631889Z INFO: Gathering HAProxy config files
2026-04-22T19:35:03.763558873Z INFO: Collecting host service logs for crio
2026-04-22T19:35:03.763974272Z INFO: Collecting host service logs for kubelet
2026-04-22T19:35:03.764317545Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T19:35:03.764486938Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T19:35:03.764676459Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T19:35:03.765025233Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T19:35:03.765367595Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T19:35:03.765709159Z INFO: Collecting host service logs for NetworkManager
2026-04-22T19:35:03.766113093Z INFO: Collecting host service logs for openvswitch
2026-04-22T19:35:03.767102482Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T19:35:03.767451980Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T19:35:03.767783980Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T19:35:03.768231267Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T19:35:03.803147218Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T19:35:03.833891648Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T19:35:04.288847143Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T19:35:04.714840502Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T19:35:04.740469949Z No resources found in openshift-etcd namespace.
2026-04-22T19:35:04.840859939Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T19:35:04.840859939Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T19:35:04.873271708Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T19:35:04.945445603Z Wrote inspect data to must-gather.
2026-04-22T19:35:05.134418147Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T19:35:05.213808974Z INFO: Worker host service log collection to complete.
2026-04-22T19:35:05.307875690Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-wprxw
2026-04-22T19:35:05.307953859Z insights-runtime-extractor-5mzdz
2026-04-22T19:35:05.307995247Z insights-runtime-extractor-9gjnl
2026-04-22T19:35:05.308016767Z insights-runtime-extractor-rdfsn
2026-04-22T19:35:05.331507525Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T19:35:05.334554587Z No resources found
2026-04-22T19:35:05.595464397Z Gathering data for ns/openshift-monitoring...
2026-04-22T19:35:05.600742745Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T19:35:05.662190351Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T19:35:05.690161194Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T19:35:05.791335158Z No resources found
2026-04-22T19:35:05.839495238Z Wrote inspect data to must-gather.
2026-04-22T19:35:05.841412402Z Wrote inspect data to must-gather.
2026-04-22T19:35:05.858392998Z Wrote inspect data to must-gather.
2026-04-22T19:35:05.911221733Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:35:05.918145443Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T19:35:05.918145443Z See 'oc rsync -h' for help and examples
2026-04-22T19:35:05.929408274Z ERROR: No running kube-apiserver pods found
2026-04-22T19:35:05.965842724Z Wrote inspect data to must-gather.
2026-04-22T19:35:06.029641830Z Wrote inspect data to must-gather.
2026-04-22T19:35:06.068357802Z INFO: 'previous-logs' folder not found on ip-10-0-134-109.ec2.internal, skipping...
2026-04-22T19:35:06.070459353Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T19:35:06.092459914Z tar: Removing leading `/' from member names
2026-04-22T19:35:06.102510796Z INFO: HAProxy config collection complete.
2026-04-22T19:35:06.139540082Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T19:35:06.164964468Z Wrote inspect data to must-gather.
2026-04-22T19:35:06.279874054Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:35:06.294677381Z Wrote inspect data to must-gather.
2026-04-22T19:35:06.346719084Z INFO: 'previous-logs' folder not found on ip-10-0-138-84.ec2.internal, skipping...
2026-04-22T19:35:06.383724392Z Wrote inspect data to must-gather.
2026-04-22T19:35:06.565245553Z Wrote inspect data to must-gather.
2026-04-22T19:35:06.602183192Z INFO: 'previous-logs' folder not found on ip-10-0-143-56.ec2.internal, skipping...
2026-04-22T19:35:06.602183192Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T19:35:06.602228011Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T19:35:06.619359396Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T19:35:06.620725942Z Wrote inspect data to must-gather.
2026-04-22T19:35:06.786470447Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T19:35:06.814211991Z Wrote inspect data to must-gather.
2026-04-22T19:35:07.021282802Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T19:35:07.025498514Z Wrote inspect data to must-gather.
2026-04-22T19:35:07.038776567Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T19:35:07.145170548Z INFO: INTERCONNECT MODE
2026-04-22T19:35:07.145209869Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T19:35:07.269238858Z Wrote inspect data to must-gather.
2026-04-22T19:35:07.322661415Z INFO: Gathering OVN_Northbound from ovnkube-node-7zbwn...
2026-04-22T19:35:07.322951524Z INFO: Gathering OVN_Northbound from ovnkube-node-p7bgj...
2026-04-22T19:35:07.323254412Z INFO: Gathering OVN_Northbound from ovnkube-node-vtpk8...
2026-04-22T19:35:07.323548869Z INFO: Gathering OVN_Southbound from ovnkube-node-7zbwn...
2026-04-22T19:35:07.323806505Z INFO: Gathering OVN_Southbound from ovnkube-node-p7bgj...
2026-04-22T19:35:07.324353046Z INFO: Gathering OVN_Southbound from ovnkube-node-vtpk8...
2026-04-22T19:35:07.553274198Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T19:35:07.605614689Z Wrote inspect data to must-gather.
2026-04-22T19:35:07.616359537Z tar: Removing leading `/' from member names
2026-04-22T19:35:07.631383937Z tar: Removing leading `/' from member names
2026-04-22T19:35:07.677171515Z tar: Removing leading `/' from member names
2026-04-22T19:35:07.677535564Z tar: Removing leading `/' from member names
2026-04-22T19:35:07.688523113Z tar: Removing leading `/' from member names
2026-04-22T19:35:07.709494925Z tar: Removing leading `/' from member names
2026-04-22T19:35:07.765479333Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:35:07.835428574Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T19:35:07.971097477Z Gathering data for ns/openshift-network-console...
2026-04-22T19:35:07.987637619Z Wrote inspect data to must-gather.
2026-04-22T19:35:08.088293958Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T19:35:08.228250273Z INFO: Waiting for network log collection to complete ...
2026-04-22T19:35:08.228351193Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T19:35:08.228657284Z INFO: Copying ovnk databases complete.
2026-04-22T19:35:08.230544081Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-22T19:35:08.235470807Z ovnk_database_store/
2026-04-22T19:35:08.235524587Z ovnk_database_store/ovnkube-node-7zbwn_sbdb
2026-04-22T19:35:08.263831462Z ovnk_database_store/ovnkube-node-7zbwn_nbdb
2026-04-22T19:35:08.276413721Z ovnk_database_store/ovnkube-node-vtpk8_sbdb
2026-04-22T19:35:08.301184562Z ovnk_database_store/ovnkube-node-vtpk8_nbdb
2026-04-22T19:35:08.309681774Z ovnk_database_store/ovnkube-node-p7bgj_nbdb
2026-04-22T19:35:08.319686000Z Wrote inspect data to must-gather.
2026-04-22T19:35:08.319949527Z ovnk_database_store/ovnkube-node-p7bgj_sbdb
2026-04-22T19:35:08.345323031Z INFO: Network log collection complete.
2026-04-22T19:35:08.351266124Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T19:35:08.426770723Z Gathering data for ns/openshift-console-operator...
2026-04-22T19:35:08.616031728Z Wrote inspect data to must-gather.
2026-04-22T19:35:08.735741886Z INFO: Getting status from alertmanager-main-0
2026-04-22T19:35:08.824424657Z Gathering data for ns/openshift-console...
2026-04-22T19:35:08.927212058Z Wrote inspect data to must-gather.
2026-04-22T19:35:09.193387608Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T19:35:09.196076038Z INFO: Fallback to identify the container image from release info
2026-04-22T19:35:09.238443461Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T19:35:09.561968410Z Gathering data for ns/openshift-dns-operator...
2026-04-22T19:35:09.746440338Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T19:35:09.841749528Z Gathering data for ns/openshift-dns...
2026-04-22T19:35:09.883365598Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T19:35:10.004555492Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T19:35:10.440801422Z Gathering data for ns/openshift-image-registry...
2026-04-22T19:35:10.859847885Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T19:35:11.134252218Z Waiting for performance profile collector pods to become ready: 2
2026-04-22T19:35:11.201431052Z Gathering data for ns/openshift-ingress...
2026-04-22T19:35:11.287436739Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:35:11.508781276Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T19:35:11.851512844Z Gathering data for ns/openshift-insights...
2026-04-22T19:35:12.257533263Z Waiting for performance profile collector pods to become ready: 3
2026-04-22T19:35:13.374340874Z Waiting for performance profile collector pods to become ready: 4
2026-04-22T19:35:13.862218323Z Gathering data for ns/kserve...
2026-04-22T19:35:14.493521526Z Waiting for performance profile collector pods to become ready: 5
2026-04-22T19:35:15.452620879Z Gathering data for ns/openshift-config...
2026-04-22T19:35:15.615315684Z Waiting for performance profile collector pods to become ready: 6
2026-04-22T19:35:15.705041521Z Gathering data for ns/openshift-config-managed...
2026-04-22T19:35:16.094661894Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T19:35:16.293703931Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:35:16.329365805Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T19:35:16.561019066Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T19:35:16.743029217Z Waiting for performance profile collector pods to become ready: 7
2026-04-22T19:35:16.834554445Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T19:35:17.098256789Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T19:35:17.343358249Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T19:35:17.615135215Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T19:35:17.875115248Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T19:35:17.925337445Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T19:35:18.153636571Z Collecting performance related data for node ip-10-0-134-109.ec2.internal
2026-04-22T19:35:18.166752000Z Collecting performance related data for node ip-10-0-138-84.ec2.internal
2026-04-22T19:35:18.173234506Z Collecting performance related data for node ip-10-0-143-56.ec2.internal
2026-04-22T19:35:18.269225914Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T19:35:18.866242396Z Gathering data for ns/openshift-multus...
2026-04-22T19:35:20.295439920Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T19:35:21.301270596Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:35:21.836417255Z Gathering data for ns/openshift-host-network...
2026-04-22T19:35:22.172241167Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T19:35:22.673764285Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T19:35:23.081454845Z Gathering data for ns/openshift-network-operator...
2026-04-22T19:35:23.454335074Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T19:35:23.709381653Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T19:35:24.134920010Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T19:35:24.406051713Z Gathering data for ns/openshift-apiserver...
2026-04-22T19:35:24.799160968Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T19:35:25.079614395Z Gathering data for ns/openshift-controller-manager...
2026-04-22T19:35:25.327262324Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T19:35:25.988056147Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T19:35:26.097223837Z Collecting kubelet logs for node ip-10-0-134-109.ec2.internal
2026-04-22T19:35:26.099050850Z Collecting kubelet logs for node ip-10-0-138-84.ec2.internal
2026-04-22T19:35:26.100710680Z Collecting kubelet logs for node ip-10-0-143-56.ec2.internal
2026-04-22T19:35:26.308579470Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:35:26.355289921Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T19:35:26.478603104Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T19:35:26.483623161Z INFO: Node performance data collection complete.
2026-04-22T19:35:26.658598983Z Gathering data for ns/openshift-service-ca...
2026-04-22T19:35:26.995796021Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T19:35:27.812168236Z Wrote inspect data to must-gather.
2026-04-22T19:35:27.812219712Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T19:35:27.812219712Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T19:35:27.950101359Z error: the server doesn't have a resource type "clusters"
2026-04-22T19:35:28.067764400Z Caches written to disk