2026-07-06T19:19:54.731367609Z [disk usage checker] Started
2026-07-06T19:19:54.734932462Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-06T19:19:54.981703965Z Gathering data for ns/openshift-cluster-version...
2026-07-06T19:19:55.143250195Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T19:19:55.270986582Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T19:19:55.375589687Z Gathering data for ns/default...
2026-07-06T19:19:55.790427984Z Gathering data for ns/openshift...
2026-07-06T19:19:56.500633225Z Gathering data for ns/kube-system...
2026-07-06T19:19:57.621484262Z Waiting on subprocesses to finish execution.
2026-07-06T19:19:57.633679572Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-06T19:19:57.634397327Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-06T19:19:57.634649840Z Executing Istio gather script
2026-07-06T19:19:57.645039534Z INFO: Gathering HAProxy config files
2026-07-06T19:19:57.650309078Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-06T19:19:57.678673943Z INFO: Collecting host service logs for crio
2026-07-06T19:19:57.679044356Z INFO: Collecting host service logs for kubelet
2026-07-06T19:19:57.679417944Z INFO: Collecting host service logs for rpm-ostreed
2026-07-06T19:19:57.679762768Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-06T19:19:57.680090052Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-06T19:19:57.680431607Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-06T19:19:57.680762090Z INFO: Collecting host service logs for NetworkManager
2026-07-06T19:19:57.681087781Z INFO: Collecting host service logs for openvswitch
2026-07-06T19:19:57.681435728Z INFO: Collecting host service logs for ovs-configuration
2026-07-06T19:19:57.681768100Z INFO: Collecting host service logs for ovsdb-server
2026-07-06T19:19:57.682099439Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-06T19:19:57.682504118Z INFO: Waiting for worker host service log collection to complete ...
2026-07-06T19:19:57.843307792Z INFO: Waiting for node performance related collection to complete ...
2026-07-06T19:19:58.490063990Z INFO: "metallb-operator" not detected. Skipping.
2026-07-06T19:19:58.538350293Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-06T19:19:58.539718680Z INFO: on-disk MachineConfig config collection complete.
2026-07-06T19:19:58.669071713Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-06T19:19:58.860100254Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-06T19:19:58.880221937Z Wrote inspect data to must-gather.
2026-07-06T19:19:59.073906230Z No resources found in openshift-etcd namespace.
2026-07-06T19:19:59.096045690Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-v2klg
2026-07-06T19:19:59.096045690Z insights-runtime-extractor-48x2h
2026-07-06T19:19:59.096045690Z insights-runtime-extractor-8h5xb
2026-07-06T19:19:59.096045690Z insights-runtime-extractor-hjkpn
2026-07-06T19:19:59.259290513Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T19:19:59.426340683Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-06T19:19:59.468714923Z No resources found
2026-07-06T19:19:59.579737339Z Gathering data for ns/openshift-etcd...
2026-07-06T19:19:59.744623822Z INFO: Worker host service log collection to complete.
2026-07-06T19:19:59.763226453Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-06T19:19:59.776710559Z Wrote inspect data to must-gather.
2026-07-06T19:19:59.777400281Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-06T19:19:59.785633111Z ERROR: No running kube-apiserver pods found
2026-07-06T19:19:59.802350506Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-06T19:19:59.879179393Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-06T19:19:59.946257534Z Wrote inspect data to must-gather.
2026-07-06T19:19:59.956885010Z No resources found
2026-07-06T19:20:00.194510697Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-06T19:20:00.196739702Z Gathering data for ns/openshift-monitoring...
2026-07-06T19:20:00.240708254Z error: the server doesn't have a resource type "performanceprofile"
2026-07-06T19:20:00.421264697Z INFO: 'previous-logs' folder not found on ip-10-0-136-29.ec2.internal, skipping...
2026-07-06T19:20:00.460609764Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-06T19:20:00.460609764Z See 'oc rsync -h' for help and examples
2026-07-06T19:20:00.535099796Z Wrote inspect data to must-gather.
2026-07-06T19:20:00.569752804Z Wrote inspect data to must-gather.
2026-07-06T19:20:00.656371109Z Wrote inspect data to must-gather.
2026-07-06T19:20:00.717168445Z Wrote inspect data to must-gather.
2026-07-06T19:20:00.795381571Z Wrote inspect data to must-gather.
2026-07-06T19:20:00.795511754Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T19:20:00.795511754Z     namespaces "assisted-installer" not found
2026-07-06T19:20:00.857636825Z Wrote inspect data to must-gather.
2026-07-06T19:20:00.884566706Z INFO: 'previous-logs' folder not found on ip-10-0-137-220.ec2.internal, skipping...
2026-07-06T19:20:00.934296372Z tar: Removing leading `/' from member names
2026-07-06T19:20:00.939236886Z INFO: HAProxy config collection complete.
2026-07-06T19:20:00.975368510Z INFO: Getting rules from prometheus-k8s-0
2026-07-06T19:20:00.998435798Z Wrote inspect data to must-gather.
2026-07-06T19:20:01.057073530Z error: the server doesn't have a resource type "machineconfigs"
2026-07-06T19:20:01.169950788Z INFO: 'previous-logs' folder not found on ip-10-0-142-220.ec2.internal, skipping...
2026-07-06T19:20:01.170009050Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-06T19:20:01.170065255Z INFO: Machine Config Daemon termination log collection complete.
2026-07-06T19:20:01.178805273Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-06T19:20:01.228602221Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-06T19:20:01.287775672Z Wrote inspect data to must-gather.
2026-07-06T19:20:01.448467702Z Wrote inspect data to must-gather.
2026-07-06T19:20:01.508505130Z INFO: Getting status/config from prometheus-k8s-0
2026-07-06T19:20:01.606654941Z INFO: INTERCONNECT MODE
2026-07-06T19:20:01.606685301Z INFO: Gathering ovn-kubernetes DBs
2026-07-06T19:20:01.625192489Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-06T19:20:01.798198406Z INFO: Gathering OVN_Northbound from ovnkube-node-4j8tj...
2026-07-06T19:20:01.798454413Z INFO: Gathering OVN_Northbound from ovnkube-node-8ltgn...
2026-07-06T19:20:01.798721578Z INFO: Gathering OVN_Northbound from ovnkube-node-mxfxf...
2026-07-06T19:20:01.799054853Z INFO: Gathering OVN_Southbound from ovnkube-node-4j8tj...
2026-07-06T19:20:01.799320590Z INFO: Gathering OVN_Southbound from ovnkube-node-8ltgn...
2026-07-06T19:20:01.799562066Z INFO: Gathering OVN_Southbound from ovnkube-node-mxfxf...
2026-07-06T19:20:01.921501237Z Wrote inspect data to must-gather.
2026-07-06T19:20:02.176813220Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-06T19:20:02.183062607Z tar: Removing leading `/' from member names
2026-07-06T19:20:02.195248920Z tar: Removing leading `/' from member names
2026-07-06T19:20:02.200343919Z tar: Removing leading `/' from member names
2026-07-06T19:20:02.253896687Z Wrote inspect data to must-gather.
2026-07-06T19:20:02.253935099Z tar: Removing leading `/' from member names
2026-07-06T19:20:02.254063035Z tar: Removing leading `/' from member names
2026-07-06T19:20:02.301347012Z tar: Removing leading `/' from member names
2026-07-06T19:20:02.495867085Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-06T19:20:02.497844561Z Wrote inspect data to must-gather.
2026-07-06T19:20:02.639538800Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-06T19:20:02.652672261Z INFO: Waiting for network log collection to complete ...
2026-07-06T19:20:02.652727062Z INFO: Waiting for ovnk database copies to complete ...
2026-07-06T19:20:02.653323634Z INFO: Copying ovnk databases complete.
2026-07-06T19:20:02.655163267Z 3.0M	must-gather/network_logs/ovnk_database_store
2026-07-06T19:20:02.657668880Z ovnk_database_store/
2026-07-06T19:20:02.657840731Z ovnk_database_store/ovnkube-node-mxfxf_sbdb
2026-07-06T19:20:02.678991331Z ovnk_database_store/ovnkube-node-8ltgn_nbdb
2026-07-06T19:20:02.686322986Z ovnk_database_store/ovnkube-node-mxfxf_nbdb
2026-07-06T19:20:02.694297214Z ovnk_database_store/ovnkube-node-4j8tj_nbdb
2026-07-06T19:20:02.703988001Z ovnk_database_store/ovnkube-node-4j8tj_sbdb
2026-07-06T19:20:02.723276517Z ovnk_database_store/ovnkube-node-8ltgn_sbdb
2026-07-06T19:20:02.729479631Z Wrote inspect data to must-gather.
2026-07-06T19:20:02.742912766Z INFO: Network log collection complete.
2026-07-06T19:20:02.779768069Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-06T19:20:02.810916580Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:02.951424744Z Wrote inspect data to must-gather.
2026-07-06T19:20:03.044599151Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:03.048320078Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-06T19:20:03.052234045Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-06T19:20:03.149365737Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T19:20:03.182768984Z Wrote inspect data to must-gather.
2026-07-06T19:20:03.268799786Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:03.432410589Z Wrote inspect data to must-gather.
2026-07-06T19:20:03.472462735Z INFO: Getting status from alertmanager-main-0
2026-07-06T19:20:03.498227129Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:03.501939324Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-06T19:20:03.724442113Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-06T19:20:03.726579439Z INFO: Fallback to identify the container image from release info
2026-07-06T19:20:03.744961628Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:03.935631168Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:03.938410330Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-06T19:20:04.012968946Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T19:20:04.204260354Z Gathering data for ns/openshift-network-console...
2026-07-06T19:20:04.222304834Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:04.426319279Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:04.428958999Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io
2026-07-06T19:20:04.429017592Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-06T19:20:04.566511729Z daemonset.apps/perf-node-gather-daemonset created
2026-07-06T19:20:04.614882962Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:04.714139879Z Waiting for performance profile collector pods to become ready: 1
2026-07-06T19:20:04.796921466Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-06T19:20:04.835864726Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:04.838940722Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io
2026-07-06T19:20:04.877603007Z Gathering data for ns/openshift-console-operator...
2026-07-06T19:20:05.032032341Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:05.247517285Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:05.250037982Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io
2026-07-06T19:20:05.375704189Z Gathering data for ns/openshift-console...
2026-07-06T19:20:05.428810321Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:05.628471052Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:05.630946222Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-06T19:20:05.806013937Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:05.847959297Z Waiting for performance profile collector pods to become ready: 2
2026-07-06T19:20:06.004942468Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:06.007687720Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-06T19:20:06.024323204Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-06T19:20:06.184232142Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:06.379304569Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:06.381513559Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-06T19:20:06.564850676Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:06.608683546Z Gathering data for ns/openshift-dns-operator...
2026-07-06T19:20:06.759414362Z Wrote inspect data to /must-gather/istio.
2026-07-06T19:20:06.974281427Z Gathering data for ns/openshift-dns...
2026-07-06T19:20:06.995487379Z Waiting for performance profile collector pods to become ready: 3
2026-07-06T19:20:07.410028256Z Done executing Istio gather script
2026-07-06T19:20:07.858450224Z Gathering data for ns/openshift-image-registry...
2026-07-06T19:20:08.107350720Z Waiting for performance profile collector pods to become ready: 4
2026-07-06T19:20:08.590857223Z Gathering data for ns/openshift-ingress-operator...
2026-07-06T19:20:09.004840135Z Gathering data for ns/openshift-ingress...
2026-07-06T19:20:09.222445488Z Waiting for performance profile collector pods to become ready: 5
2026-07-06T19:20:09.400293295Z Gathering data for ns/openshift-ingress-canary...
2026-07-06T19:20:09.803094506Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-06T19:20:09.985085178Z Gathering data for ns/openshift-insights...
2026-07-06T19:20:10.332624667Z Waiting for performance profile collector pods to become ready: 6
2026-07-06T19:20:11.457512089Z Waiting for performance profile collector pods to become ready: 7
2026-07-06T19:20:12.578964292Z Waiting for performance profile collector pods to become ready: 8
2026-07-06T19:20:12.696971879Z Gathering data for ns/kserve...
2026-07-06T19:20:13.701823373Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-06T19:20:13.999059419Z Collecting performance related data for node ip-10-0-136-29.ec2.internal
2026-07-06T19:20:14.006427361Z Collecting performance related data for node ip-10-0-137-220.ec2.internal
2026-07-06T19:20:14.013819699Z Collecting performance related data for node ip-10-0-142-220.ec2.internal
2026-07-06T19:20:14.331509634Z Gathering data for ns/openshift-config...
2026-07-06T19:20:14.730443789Z Gathering data for ns/openshift-config-managed...
2026-07-06T19:20:14.808659261Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-06T19:20:15.254182597Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-06T19:20:15.573397596Z Gathering data for ns/openshift-kube-apiserver...
2026-07-06T19:20:15.925085615Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-06T19:20:16.220878608Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-06T19:20:16.528622395Z Gathering data for ns/openshift-kube-scheduler...
2026-07-06T19:20:16.840771622Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-06T19:20:17.223518452Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-06T19:20:17.661293585Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-06T19:20:17.980348452Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-06T19:20:18.506870555Z Gathering data for ns/openshift-multus...
2026-07-06T19:20:19.813053306Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-06T19:20:20.150941486Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-06T19:20:21.661776603Z Gathering data for ns/openshift-host-network...
2026-07-06T19:20:22.025720692Z Gathering data for ns/openshift-network-diagnostics...
2026-07-06T19:20:22.332149649Z Collecting kubelet logs for node ip-10-0-136-29.ec2.internal
2026-07-06T19:20:22.333857970Z Collecting kubelet logs for node ip-10-0-137-220.ec2.internal
2026-07-06T19:20:22.335660536Z Collecting kubelet logs for node ip-10-0-142-220.ec2.internal
2026-07-06T19:20:22.705828013Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-czgnz namespace
2026-07-06T19:20:22.711428793Z INFO: Node performance data collection complete.
2026-07-06T19:20:22.762541860Z Gathering data for ns/openshift-network-node-identity...
2026-07-06T19:20:23.196955097Z Gathering data for ns/openshift-network-operator...
2026-07-06T19:20:23.626473002Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-06T19:20:23.878106944Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-06T19:20:24.385792049Z Gathering data for ns/openshift-apiserver-operator...
2026-07-06T19:20:24.691855326Z Gathering data for ns/openshift-apiserver...
2026-07-06T19:20:24.817842211Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-06T19:20:25.068354248Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-06T19:20:25.355204552Z Gathering data for ns/openshift-controller-manager...
2026-07-06T19:20:25.676662696Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-06T19:20:26.342370882Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-06T19:20:26.634785588Z Gathering data for ns/openshift-service-ca-operator...
2026-07-06T19:20:27.021137994Z Gathering data for ns/openshift-service-ca...
2026-07-06T19:20:27.388949856Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-06T19:20:28.290201135Z Wrote inspect data to must-gather.
2026-07-06T19:20:28.290242722Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T19:20:28.290242722Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-06T19:20:28.419025582Z error: the server doesn't have a resource type "clusters"
2026-07-06T19:20:28.867164347Z Caches written to disk