2026-08-04T18:55:59.405696684Z [disk usage checker] Started
2026-08-04T18:55:59.409053444Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:55:59.756438998Z Gathering data for ns/openshift-cluster-version...
2026-08-04T18:55:59.946109844Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-04T18:56:00.083529621Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-04T18:56:00.198821911Z Gathering data for ns/default...
2026-08-04T18:56:00.698655784Z Gathering data for ns/openshift...
2026-08-04T18:56:01.271713635Z Gathering data for ns/kube-system...
2026-08-04T18:56:02.451968261Z Gathering data for ns/openshift-etcd...
2026-08-04T18:56:02.592616413Z Waiting on subprocesses to finish execution.
2026-08-04T18:56:02.600952722Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-08-04T18:56:02.606707766Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-08-04T18:56:02.610019113Z INFO: Gathering HAProxy config files
2026-08-04T18:56:02.612652723Z Executing Istio gather script
2026-08-04T18:56:02.616706455Z INFO: Gathering machine config daemon's old logs from all nodes
2026-08-04T18:56:02.665722306Z INFO: Collecting host service logs for crio
2026-08-04T18:56:02.666103598Z INFO: Collecting host service logs for kubelet
2026-08-04T18:56:02.666526167Z INFO: Collecting host service logs for rpm-ostreed
2026-08-04T18:56:02.666923605Z INFO: Collecting host service logs for ostree-finalize-staged
2026-08-04T18:56:02.667294047Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-08-04T18:56:02.667664201Z INFO: Collecting host service logs for machine-config-daemon-host
2026-08-04T18:56:02.668045955Z INFO: Collecting host service logs for NetworkManager
2026-08-04T18:56:02.668438222Z INFO: Collecting host service logs for openvswitch
2026-08-04T18:56:02.668804411Z INFO: Collecting host service logs for ovs-configuration
2026-08-04T18:56:02.669177096Z INFO: Collecting host service logs for ovsdb-server
2026-08-04T18:56:02.670973306Z INFO: Collecting host service logs for ovs-vswitchd
2026-08-04T18:56:02.672000140Z INFO: Waiting for worker host service log collection to complete ...
2026-08-04T18:56:02.715654647Z INFO: Waiting for node performance related collection to complete ...
2026-08-04T18:56:03.083189824Z INFO: Collecting Insights Archives from insights-operator-77696685b-zxjxn
2026-08-04T18:56:03.083240002Z insights-runtime-extractor-dkvs8
2026-08-04T18:56:03.083259441Z insights-runtime-extractor-f767j
2026-08-04T18:56:03.083280590Z insights-runtime-extractor-rlzgv
2026-08-04T18:56:03.215631683Z No resources found in openshift-etcd namespace.
2026-08-04T18:56:03.538448075Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-08-04T18:56:03.538448075Z See 'oc rsync -h' for help and examples
2026-08-04T18:56:03.767956914Z Wrote inspect data to must-gather.
2026-08-04T18:56:03.768048787Z error: inspection completed with the errors occurred while gathering data:
2026-08-04T18:56:03.768048787Z     namespaces "assisted-installer" not found
2026-08-04T18:56:03.932715070Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-08-04T18:56:03.932808883Z INFO: on-disk MachineConfig config collection complete.
2026-08-04T18:56:03.987931069Z No resources found
2026-08-04T18:56:04.045016982Z INFO: "metallb-operator" not detected. Skipping.
2026-08-04T18:56:04.369532450Z INFO: Found 1 replicas - prometheus-k8s-0
2026-08-04T18:56:04.507964891Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-08-04T18:56:04.512656384Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-08-04T18:56:04.586743866Z Wrote inspect data to must-gather.
2026-08-04T18:56:04.621938505Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:56:04.670656650Z INFO: Waiting for HAProxy config collection to complete ...
2026-08-04T18:56:04.759190477Z INFO: Worker host service log collection to complete.
2026-08-04T18:56:04.855942549Z No resources found
2026-08-04T18:56:04.887364519Z error: the server doesn't have a resource type "performanceprofile"
2026-08-04T18:56:04.959603925Z INFO: "sriov-network-operator" not detected. Skipping.
2026-08-04T18:56:04.960778995Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-04T18:56:04.995948115Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.099880872Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.208454538Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.220826005Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.300800621Z Gathering data for ns/openshift-monitoring...
2026-08-04T18:56:05.384308815Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.386834236Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-08-04T18:56:05.402615059Z tar: Removing leading `/' from member names
2026-08-04T18:56:05.409928284Z INFO: HAProxy config collection complete.
2026-08-04T18:56:05.429022952Z ERROR: No running kube-apiserver pods found
2026-08-04T18:56:05.441462299Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-08-04T18:56:05.527279870Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.529995158Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.571088881Z INFO: 'previous-logs' folder not found on ip-10-0-129-212.ec2.internal, skipping...
2026-08-04T18:56:05.703765753Z error: the server doesn't have a resource type "machineconfigs"
2026-08-04T18:56:05.858624823Z Wrote inspect data to must-gather.
2026-08-04T18:56:05.884918130Z INFO: 'previous-logs' folder not found on ip-10-0-132-159.ec2.internal, skipping...
2026-08-04T18:56:05.929642578Z error: the server doesn't have a resource type "machineconfigpools"
2026-08-04T18:56:05.982227267Z INFO: Getting rules from prometheus-k8s-0
2026-08-04T18:56:06.205959428Z INFO: 'previous-logs' folder not found on ip-10-0-132-242.ec2.internal, skipping...
2026-08-04T18:56:06.205996352Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-08-04T18:56:06.205996352Z INFO: Machine Config Daemon termination log collection complete.
2026-08-04T18:56:06.206637196Z Wrote inspect data to must-gather.
2026-08-04T18:56:06.230905728Z Wrote inspect data to must-gather.
2026-08-04T18:56:06.414942845Z error: the server doesn't have a resource type "kubeletconfigs"
2026-08-04T18:56:06.489758871Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-08-04T18:56:06.514782475Z INFO: Getting status/config from prometheus-k8s-0
2026-08-04T18:56:06.550416914Z INFO: INTERCONNECT MODE
2026-08-04T18:56:06.550466217Z INFO: Gathering ovn-kubernetes DBs
2026-08-04T18:56:06.663394725Z Wrote inspect data to must-gather.
2026-08-04T18:56:06.737051788Z INFO: Gathering OVN_Northbound from ovnkube-node-hs8zb...
2026-08-04T18:56:06.737331644Z INFO: Gathering OVN_Northbound from ovnkube-node-k2wfn...
2026-08-04T18:56:06.737659541Z INFO: Gathering OVN_Northbound from ovnkube-node-kfzm2...
2026-08-04T18:56:06.737992833Z INFO: Gathering OVN_Southbound from ovnkube-node-hs8zb...
2026-08-04T18:56:06.738297241Z INFO: Gathering OVN_Southbound from ovnkube-node-k2wfn...
2026-08-04T18:56:06.738713090Z INFO: Gathering OVN_Southbound from ovnkube-node-kfzm2...
2026-08-04T18:56:07.014845758Z Wrote inspect data to must-gather.
2026-08-04T18:56:07.086596364Z tar: Removing leading `/' from member names
2026-08-04T18:56:07.103911283Z INFO: Getting status/flags from prometheus-k8s-0
2026-08-04T18:56:07.129570028Z tar: Removing leading `/' from member names
2026-08-04T18:56:07.129857222Z tar: Removing leading `/' from member names
2026-08-04T18:56:07.138045909Z tar: Removing leading `/' from member names
2026-08-04T18:56:07.140846560Z tar: Removing leading `/' from member names
2026-08-04T18:56:07.161297637Z tar: Removing leading `/' from member names
2026-08-04T18:56:07.289315758Z Wrote inspect data to must-gather.
2026-08-04T18:56:07.425751308Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-08-04T18:56:07.540665482Z Wrote inspect data to must-gather.
2026-08-04T18:56:07.627382725Z INFO: Waiting for network log collection to complete ...
2026-08-04T18:56:07.627538597Z INFO: Waiting for ovnk database copies to complete ...
2026-08-04T18:56:07.628016552Z INFO: Copying ovnk databases complete.
2026-08-04T18:56:07.630151170Z 4.6M	must-gather/network_logs/ovnk_database_store
2026-08-04T18:56:07.632530226Z ovnk_database_store/
2026-08-04T18:56:07.632530226Z ovnk_database_store/ovnkube-node-kfzm2_sbdb
2026-08-04T18:56:07.653588967Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-08-04T18:56:07.662387656Z ovnk_database_store/ovnkube-node-k2wfn_sbdb
2026-08-04T18:56:07.691303249Z ovnk_database_store/ovnkube-node-kfzm2_nbdb
2026-08-04T18:56:07.703122258Z ovnk_database_store/ovnkube-node-hs8zb_sbdb
2026-08-04T18:56:07.722831253Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-08-04T18:56:07.731265485Z ovnk_database_store/ovnkube-node-hs8zb_nbdb
2026-08-04T18:56:07.741313539Z ovnk_database_store/ovnkube-node-k2wfn_nbdb
2026-08-04T18:56:07.757315353Z INFO: Network log collection complete.
2026-08-04T18:56:07.772000641Z Wrote inspect data to must-gather.
2026-08-04T18:56:07.911971496Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:08.017104121Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-04T18:56:08.018181791Z Wrote inspect data to must-gather.
2026-08-04T18:56:08.095412640Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-08-04T18:56:08.131023933Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:08.134737742Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-08-04T18:56:08.262541495Z Wrote inspect data to must-gather.
2026-08-04T18:56:08.371604743Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:08.580827534Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-08-04T18:56:08.584657208Z INFO: Fallback to identify the container image from release info
2026-08-04T18:56:08.597508129Z INFO: Getting status from alertmanager-main-0
2026-08-04T18:56:08.613842251Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:08.618030750Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-08-04T18:56:08.683499610Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-04T18:56:08.842631697Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:08.882548837Z Gathering data for ns/openshift-network-console...
2026-08-04T18:56:09.049392182Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:09.052277505Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-08-04T18:56:09.284871775Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:09.355958354Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-08-04T18:56:09.379729254Z Gathering data for ns/openshift-console-operator...
2026-08-04T18:56:09.515305938Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:09.517444767Z daemonset.apps/perf-node-gather-daemonset created
2026-08-04T18:56:09.518394172Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-08-04T18:56:09.637134798Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:56:09.661997644Z Waiting for performance profile collector pods to become ready: 1
2026-08-04T18:56:09.712994224Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:09.874787360Z Gathering data for ns/openshift-console...
2026-08-04T18:56:09.931457572Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:09.935501042Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-08-04T18:56:10.139345533Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:10.336475353Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:10.339438738Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-08-04T18:56:10.541690230Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:10.607335077Z Gathering data for ns/openshift-cluster-storage-operator...
2026-08-04T18:56:10.769686713Z Wrote inspect data to /must-gather/istio.
2026-08-04T18:56:10.819521496Z Waiting for performance profile collector pods to become ready: 2
2026-08-04T18:56:11.259334935Z Gathering data for ns/openshift-dns-operator...
2026-08-04T18:56:11.499290658Z Done executing Istio gather script
2026-08-04T18:56:11.620615153Z Gathering data for ns/openshift-dns...
2026-08-04T18:56:11.931852415Z Waiting for performance profile collector pods to become ready: 3
2026-08-04T18:56:12.457281841Z Gathering data for ns/openshift-image-registry...
2026-08-04T18:56:13.021977335Z Gathering data for ns/openshift-ingress-operator...
2026-08-04T18:56:13.068786369Z Waiting for performance profile collector pods to become ready: 4
2026-08-04T18:56:13.406156155Z Gathering data for ns/openshift-ingress...
2026-08-04T18:56:13.819275086Z Gathering data for ns/openshift-ingress-canary...
2026-08-04T18:56:14.194760246Z Waiting for performance profile collector pods to become ready: 5
2026-08-04T18:56:14.330344744Z Gathering data for ns/openshift-insights...
2026-08-04T18:56:14.645912207Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:56:15.316042408Z Waiting for performance profile collector pods to become ready: 6
2026-08-04T18:56:16.447060955Z Waiting for performance profile collector pods to become ready: 7
2026-08-04T18:56:16.897383031Z Gathering data for ns/kserve...
2026-08-04T18:56:17.610917782Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-08-04T18:56:17.893915445Z Collecting performance related data for node ip-10-0-132-242.ec2.internal
2026-08-04T18:56:17.900791736Z Collecting performance related data for node ip-10-0-132-159.ec2.internal
2026-08-04T18:56:17.909216932Z Collecting performance related data for node ip-10-0-129-212.ec2.internal
2026-08-04T18:56:19.651945834Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:56:19.738108969Z Gathering data for ns/openshift-config...
2026-08-04T18:56:20.073328634Z Gathering data for ns/openshift-config-managed...
2026-08-04T18:56:20.602887958Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-08-04T18:56:20.976997018Z Gathering data for ns/openshift-kube-apiserver...
2026-08-04T18:56:21.333230030Z Gathering data for ns/openshift-kube-controller-manager...
2026-08-04T18:56:21.676143953Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-08-04T18:56:22.048640194Z Gathering data for ns/openshift-kube-scheduler...
2026-08-04T18:56:22.447608443Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-08-04T18:56:22.870314057Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-08-04T18:56:23.378152701Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-08-04T18:56:23.746381703Z Gathering data for ns/openshift-user-workload-monitoring...
2026-08-04T18:56:24.321473495Z Gathering data for ns/openshift-multus...
2026-08-04T18:56:24.657183792Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:56:26.115517854Z Gathering data for ns/openshift-ovn-kubernetes...
2026-08-04T18:56:26.462875583Z Collecting kubelet logs for node ip-10-0-132-242.ec2.internal
2026-08-04T18:56:26.464746842Z Collecting kubelet logs for node ip-10-0-132-159.ec2.internal
2026-08-04T18:56:26.466332061Z Collecting kubelet logs for node ip-10-0-129-212.ec2.internal
2026-08-04T18:56:26.900299420Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vp9hh namespace
2026-08-04T18:56:26.906107434Z INFO: Node performance data collection complete.
2026-08-04T18:56:27.743465374Z Gathering data for ns/openshift-host-network...
2026-08-04T18:56:28.122420665Z Gathering data for ns/openshift-network-diagnostics...
2026-08-04T18:56:28.712030274Z Gathering data for ns/openshift-network-node-identity...
2026-08-04T18:56:29.210562365Z Gathering data for ns/openshift-network-operator...
2026-08-04T18:56:29.651325020Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-08-04T18:56:29.663270315Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:56:29.961239046Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-08-04T18:56:30.589765128Z Gathering data for ns/openshift-apiserver-operator...
2026-08-04T18:56:30.946224259Z Gathering data for ns/openshift-apiserver...
2026-08-04T18:56:31.588550180Z Gathering data for ns/openshift-controller-manager-operator...
2026-08-04T18:56:31.901286497Z Gathering data for ns/openshift-controller-manager...
2026-08-04T18:56:32.249227841Z Gathering data for ns/openshift-cluster-samples-operator...
2026-08-04T18:56:32.987848463Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-08-04T18:56:33.325932715Z Gathering data for ns/openshift-service-ca-operator...
2026-08-04T18:56:33.688439729Z Gathering data for ns/openshift-service-ca...
2026-08-04T18:56:34.120399598Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-08-04T18:56:34.668200987Z [disk usage checker] Volume usage percentage: current = 44 ; allowed = 70
2026-08-04T18:56:35.161565294Z Wrote inspect data to must-gather.
2026-08-04T18:56:35.161628598Z error: inspection completed with the errors occurred while gathering data:
2026-08-04T18:56:35.161628598Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-08-04T18:56:35.305000464Z error: the server doesn't have a resource type "clusters"
2026-08-04T18:56:35.598837814Z Caches written to disk