2026-07-22T16:43:46.411830402Z [disk usage checker] Started
2026-07-22T16:43:46.415533955Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-22T16:43:46.614433457Z Gathering data for ns/openshift-cluster-version...
2026-07-22T16:43:46.761693706Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-22T16:43:46.853494581Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-22T16:43:46.937360039Z Gathering data for ns/default...
2026-07-22T16:43:47.260147732Z Gathering data for ns/openshift...
2026-07-22T16:43:47.747299862Z Gathering data for ns/kube-system...
2026-07-22T16:43:48.576412590Z Gathering data for ns/openshift-etcd...
2026-07-22T16:43:48.928033134Z Wrote inspect data to must-gather.
2026-07-22T16:43:48.928078621Z error: inspection completed with the errors occurred while gathering data:
2026-07-22T16:43:48.928078621Z     namespaces "assisted-installer" not found
2026-07-22T16:43:49.023681625Z Waiting on subprocesses to finish execution.
2026-07-22T16:43:49.031295933Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-22T16:43:49.031725813Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-22T16:43:49.036067848Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-22T16:43:49.044362437Z INFO: Gathering HAProxy config files
2026-07-22T16:43:49.050411374Z Executing Istio gather script
2026-07-22T16:43:49.067829803Z INFO: Collecting host service logs for crio
2026-07-22T16:43:49.068178883Z INFO: Collecting host service logs for kubelet
2026-07-22T16:43:49.068481665Z INFO: Collecting host service logs for rpm-ostreed
2026-07-22T16:43:49.068778269Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-22T16:43:49.069726907Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-22T16:43:49.070109613Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-22T16:43:49.070496704Z INFO: Collecting host service logs for NetworkManager
2026-07-22T16:43:49.071201259Z INFO: Collecting host service logs for openvswitch
2026-07-22T16:43:49.071575441Z INFO: Collecting host service logs for ovs-configuration
2026-07-22T16:43:49.071975906Z INFO: Collecting host service logs for ovsdb-server
2026-07-22T16:43:49.072346578Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-22T16:43:49.072751995Z INFO: Waiting for worker host service log collection to complete ...
2026-07-22T16:43:49.498102101Z INFO: Waiting for node performance related collection to complete ...
2026-07-22T16:43:49.609082223Z INFO: "metallb-operator" not detected. Skipping.
2026-07-22T16:43:49.624483206Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-22T16:43:49.898620082Z No resources found
2026-07-22T16:43:49.908107126Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-22T16:43:50.060211200Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-22T16:43:50.278233215Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-qmch7
2026-07-22T16:43:50.278233215Z insights-runtime-extractor-6qnpq
2026-07-22T16:43:50.278233215Z insights-runtime-extractor-kg5b9
2026-07-22T16:43:50.278233215Z insights-runtime-extractor-mgbrd
2026-07-22T16:43:50.344938422Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-22T16:43:50.382625452Z No resources found in openshift-etcd namespace.
2026-07-22T16:43:50.544587118Z Wrote inspect data to must-gather.
2026-07-22T16:43:50.587102627Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-22T16:43:50.587174487Z INFO: on-disk MachineConfig config collection complete.
2026-07-22T16:43:50.626759013Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-22T16:43:50.773314643Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-22T16:43:50.785991664Z No resources found
2026-07-22T16:43:50.847456352Z error: the server doesn't have a resource type "performanceprofile"
2026-07-22T16:43:50.897538338Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-22T16:43:50.897538338Z See 'oc rsync -h' for help and examples
2026-07-22T16:43:50.908792234Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-22T16:43:51.011757558Z INFO: Worker host service log collection to complete.
2026-07-22T16:43:51.082007010Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.226879972Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.243658031Z tar: Removing leading `/' from member names
2026-07-22T16:43:51.247535675Z INFO: HAProxy config collection complete.
2026-07-22T16:43:51.271491190Z ERROR: No running kube-apiserver pods found
2026-07-22T16:43:51.281450972Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.319754840Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.374224732Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-22T16:43:51.430877924Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-22T16:43:51.432372326Z INFO: 'previous-logs' folder not found on ip-10-0-129-160.ec2.internal, skipping...
2026-07-22T16:43:51.517997445Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.571749535Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.583792613Z Gathering data for ns/openshift-monitoring...
2026-07-22T16:43:51.643881153Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.752491507Z error: the server doesn't have a resource type "machineconfigs"
2026-07-22T16:43:51.760421806Z INFO: 'previous-logs' folder not found on ip-10-0-142-1.ec2.internal, skipping...
2026-07-22T16:43:51.882489470Z INFO: Getting rules from prometheus-k8s-0
2026-07-22T16:43:51.929721404Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-22T16:43:51.935836575Z Wrote inspect data to must-gather.
2026-07-22T16:43:51.943090039Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-22T16:43:52.083692382Z INFO: 'previous-logs' folder not found on ip-10-0-142-205.ec2.internal, skipping...
2026-07-22T16:43:52.083769780Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-22T16:43:52.083801472Z INFO: Machine Config Daemon termination log collection complete.
2026-07-22T16:43:52.181459666Z Wrote inspect data to must-gather.
2026-07-22T16:43:52.191149342Z Wrote inspect data to must-gather.
2026-07-22T16:43:52.299955650Z INFO: Getting status/config from prometheus-k8s-0
2026-07-22T16:43:52.331582715Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-22T16:43:52.472198376Z INFO: INTERCONNECT MODE
2026-07-22T16:43:52.472238038Z INFO: Gathering ovn-kubernetes DBs
2026-07-22T16:43:52.540861589Z Wrote inspect data to must-gather.
2026-07-22T16:43:52.550610781Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-22T16:43:52.617628633Z INFO: Gathering OVN_Northbound from ovnkube-node-4ln74...
2026-07-22T16:43:52.617844938Z INFO: Gathering OVN_Northbound from ovnkube-node-87d4v...
2026-07-22T16:43:52.618175506Z INFO: Gathering OVN_Northbound from ovnkube-node-xjtvj...
2026-07-22T16:43:52.618454349Z INFO: Gathering OVN_Southbound from ovnkube-node-4ln74...
2026-07-22T16:43:52.618768732Z INFO: Gathering OVN_Southbound from ovnkube-node-87d4v...
2026-07-22T16:43:52.619093804Z INFO: Gathering OVN_Southbound from ovnkube-node-xjtvj...
2026-07-22T16:43:52.825265978Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-22T16:43:52.861915142Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:52.883638681Z Wrote inspect data to must-gather.
2026-07-22T16:43:52.909412209Z tar: Removing leading `/' from member names
2026-07-22T16:43:52.923335976Z tar: Removing leading `/' from member names
2026-07-22T16:43:52.965395518Z tar: Removing leading `/' from member names
2026-07-22T16:43:52.965592028Z tar: Removing leading `/' from member names
2026-07-22T16:43:52.974354566Z tar: Removing leading `/' from member names
2026-07-22T16:43:53.007160710Z tar: Removing leading `/' from member names
2026-07-22T16:43:53.117109137Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:53.117984597Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-22T16:43:53.119859451Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-22T16:43:53.149117393Z Wrote inspect data to must-gather.
2026-07-22T16:43:53.359022026Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:53.363562328Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-22T16:43:53.385100690Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-22T16:43:53.397067344Z Wrote inspect data to must-gather.
2026-07-22T16:43:53.476358846Z INFO: Waiting for network log collection to complete ...
2026-07-22T16:43:53.476393340Z INFO: Waiting for ovnk database copies to complete ...
2026-07-22T16:43:53.476768102Z INFO: Copying ovnk databases complete.
2026-07-22T16:43:53.478342958Z 3.5M	must-gather/network_logs/ovnk_database_store
2026-07-22T16:43:53.480852742Z ovnk_database_store/
2026-07-22T16:43:53.480930325Z ovnk_database_store/ovnkube-node-4ln74_nbdb
2026-07-22T16:43:53.492569529Z ovnk_database_store/ovnkube-node-xjtvj_nbdb
2026-07-22T16:43:53.506102171Z ovnk_database_store/ovnkube-node-xjtvj_sbdb
2026-07-22T16:43:53.528506642Z ovnk_database_store/ovnkube-node-87d4v_nbdb
2026-07-22T16:43:53.538841464Z ovnk_database_store/ovnkube-node-4ln74_sbdb
2026-07-22T16:43:53.558753810Z ovnk_database_store/ovnkube-node-87d4v_sbdb
2026-07-22T16:43:53.582499969Z INFO: Network log collection complete.
2026-07-22T16:43:53.598595730Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:53.601345301Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-22T16:43:53.637451711Z Wrote inspect data to must-gather.
2026-07-22T16:43:53.680779320Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-22T16:43:53.805872335Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:53.841698097Z Wrote inspect data to must-gather.
2026-07-22T16:43:54.015651575Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:54.017979945Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-22T16:43:54.019778576Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-22T16:43:54.067948870Z INFO: Getting status from alertmanager-main-0
2026-07-22T16:43:54.139812063Z Wrote inspect data to must-gather.
2026-07-22T16:43:54.218307781Z Gathering data for ns/openshift-network-console...
2026-07-22T16:43:54.256878997Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:54.405726467Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-22T16:43:54.407692761Z INFO: Fallback to identify the container image from release info
2026-07-22T16:43:54.454793632Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:54.457588178Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-22T16:43:54.630093400Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:54.687334152Z Gathering data for ns/openshift-console-operator...
2026-07-22T16:43:54.821805255Z Wrote inspect data to /must-gather/istio.
2026-07-22T16:43:55.101218519Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-22T16:43:55.108281714Z Gathering data for ns/openshift-console...
2026-07-22T16:43:55.233345400Z daemonset.apps/perf-node-gather-daemonset created
2026-07-22T16:43:55.361595998Z Waiting for performance profile collector pods to become ready: 1
2026-07-22T16:43:55.493100526Z Done executing Istio gather script
2026-07-22T16:43:55.656271502Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-22T16:43:56.176418531Z Gathering data for ns/openshift-dns-operator...
2026-07-22T16:43:56.438021242Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-22T16:43:56.482105484Z Waiting for performance profile collector pods to become ready: 2
2026-07-22T16:43:56.490589026Z Gathering data for ns/openshift-dns...
2026-07-22T16:43:57.151798547Z Gathering data for ns/openshift-image-registry...
2026-07-22T16:43:57.597870030Z Waiting for performance profile collector pods to become ready: 3
2026-07-22T16:43:57.651529249Z Gathering data for ns/openshift-ingress-operator...
2026-07-22T16:43:57.974392136Z Gathering data for ns/openshift-ingress...
2026-07-22T16:43:58.343933391Z Gathering data for ns/openshift-ingress-canary...
2026-07-22T16:43:58.719936732Z Waiting for performance profile collector pods to become ready: 4
2026-07-22T16:43:58.748766956Z Gathering data for ns/openshift-insights...
2026-07-22T16:43:59.830747454Z Waiting for performance profile collector pods to become ready: 5
2026-07-22T16:44:00.846804888Z Gathering data for ns/openshift-config...
2026-07-22T16:44:00.951381688Z Waiting for performance profile collector pods to become ready: 6
2026-07-22T16:44:01.132150291Z Gathering data for ns/openshift-config-managed...
2026-07-22T16:44:01.443447873Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-22T16:44:01.570558369Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-22T16:44:01.835063124Z Gathering data for ns/openshift-kube-apiserver...
2026-07-22T16:44:02.072519051Z Waiting for performance profile collector pods to become ready: 7
2026-07-22T16:44:02.121777075Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-22T16:44:02.464788859Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-22T16:44:02.758431188Z Gathering data for ns/openshift-kube-scheduler...
2026-07-22T16:44:03.067602899Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-22T16:44:03.192668578Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-22T16:44:03.430831273Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-22T16:44:03.442456169Z Collecting performance related data for node ip-10-0-142-1.ec2.internal
2026-07-22T16:44:03.450285830Z Collecting performance related data for node ip-10-0-129-160.ec2.internal
2026-07-22T16:44:03.458827978Z Collecting performance related data for node ip-10-0-142-205.ec2.internal
2026-07-22T16:44:03.854679287Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-22T16:44:04.268053976Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-22T16:44:04.881902578Z Gathering data for ns/openshift-multus...
2026-07-22T16:44:06.448117298Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-22T16:44:06.523722323Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-22T16:44:07.885076671Z Gathering data for ns/openshift-host-network...
2026-07-22T16:44:08.218960363Z Gathering data for ns/openshift-network-diagnostics...
2026-07-22T16:44:08.834660680Z Gathering data for ns/openshift-network-node-identity...
2026-07-22T16:44:09.262023777Z Gathering data for ns/openshift-network-operator...
2026-07-22T16:44:09.679002830Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-22T16:44:09.960573872Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-22T16:44:10.460928408Z Gathering data for ns/openshift-apiserver-operator...
2026-07-22T16:44:10.770048510Z Gathering data for ns/openshift-apiserver...
2026-07-22T16:44:11.168069801Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-22T16:44:11.447683415Z Gathering data for ns/openshift-controller-manager...
2026-07-22T16:44:11.453716785Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-22T16:44:11.534038860Z Collecting kubelet logs for node ip-10-0-142-1.ec2.internal
2026-07-22T16:44:11.535850889Z Collecting kubelet logs for node ip-10-0-129-160.ec2.internal
2026-07-22T16:44:11.537629358Z Collecting kubelet logs for node ip-10-0-142-205.ec2.internal
2026-07-22T16:44:11.768291916Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-22T16:44:11.881710142Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-wwm22 namespace
2026-07-22T16:44:11.887944029Z INFO: Node performance data collection complete.
2026-07-22T16:44:12.488799794Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-22T16:44:12.836733100Z Gathering data for ns/openshift-service-ca-operator...
2026-07-22T16:44:13.159954173Z Gathering data for ns/openshift-service-ca...
2026-07-22T16:44:13.549437727Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-22T16:44:14.486112210Z Wrote inspect data to must-gather.
2026-07-22T16:44:14.486161535Z error: inspection completed with the errors occurred while gathering data:
2026-07-22T16:44:14.486161535Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-22T16:44:14.610437669Z error: the server doesn't have a resource type "clusters"
2026-07-22T16:44:14.741578311Z Caches written to disk