2026-06-09T04:30:59.757175679Z [disk usage checker] Started
2026-06-09T04:30:59.760928041Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T04:30:59.948348382Z Gathering data for ns/openshift-cluster-version...
2026-06-09T04:31:00.080338878Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T04:31:00.154114515Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T04:31:00.228860924Z Gathering data for ns/default...
2026-06-09T04:31:00.486337621Z Gathering data for ns/openshift...
2026-06-09T04:31:00.906579811Z Gathering data for ns/kube-system...
2026-06-09T04:31:01.512660126Z Gathering data for ns/openshift-etcd...
2026-06-09T04:31:01.793039418Z Wrote inspect data to must-gather.
2026-06-09T04:31:01.793069875Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T04:31:01.793069875Z     namespaces "assisted-installer" not found
2026-06-09T04:31:02.300555453Z Waiting on subprocesses to finish execution.
2026-06-09T04:31:02.309595879Z Executing Istio gather script
2026-06-09T04:31:02.312201107Z INFO: Gathering HAProxy config files
2026-06-09T04:31:02.314798158Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-09T04:31:02.314989935Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-09T04:31:02.321772737Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-09T04:31:02.322797125Z INFO: Waiting for node performance related collection to complete ...
2026-06-09T04:31:02.324388207Z INFO: Collecting host service logs for crio
2026-06-09T04:31:02.324712868Z INFO: Collecting host service logs for kubelet
2026-06-09T04:31:02.325031257Z INFO: Collecting host service logs for rpm-ostreed
2026-06-09T04:31:02.325335660Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-09T04:31:02.325676622Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-09T04:31:02.326002803Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-09T04:31:02.326365119Z INFO: Collecting host service logs for NetworkManager
2026-06-09T04:31:02.326702632Z INFO: Collecting host service logs for openvswitch
2026-06-09T04:31:02.327015182Z INFO: Collecting host service logs for ovs-configuration
2026-06-09T04:31:02.327371044Z INFO: Collecting host service logs for ovsdb-server
2026-06-09T04:31:02.327695164Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-09T04:31:02.328049085Z INFO: Waiting for worker host service log collection to complete ...
2026-06-09T04:31:03.367716195Z Wrote inspect data to must-gather.
2026-06-09T04:31:03.374750716Z INFO: "metallb-operator" not detected. Skipping.
2026-06-09T04:31:03.385409233Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-09T04:31:03.404577995Z INFO: Collecting Insights Archives from insights-operator-896549976-rpgsb
2026-06-09T04:31:03.404648055Z insights-runtime-extractor-2bxbh
2026-06-09T04:31:03.404667591Z insights-runtime-extractor-jnmm2
2026-06-09T04:31:03.404700677Z insights-runtime-extractor-krns8
2026-06-09T04:31:03.413812791Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-09T04:31:03.572459589Z error: the server doesn't have a resource type "performanceprofile"
2026-06-09T04:31:03.601837085Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-09T04:31:03.601924695Z INFO: on-disk MachineConfig config collection complete.
2026-06-09T04:31:03.875184579Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-09T04:31:03.919231200Z Wrote inspect data to must-gather.
2026-06-09T04:31:03.919613016Z No resources found
2026-06-09T04:31:03.931870053Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-09T04:31:03.952324546Z INFO: Worker host service log collection to complete.
2026-06-09T04:31:03.952926817Z No resources found in openshift-etcd namespace.
2026-06-09T04:31:03.967686930Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-09T04:31:04.168297402Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-09T04:31:04.168297402Z See 'oc rsync -h' for help and examples
2026-06-09T04:31:04.488408538Z ERROR: No running kube-apiserver pods found
2026-06-09T04:31:04.540545003Z Gathering data for ns/openshift-monitoring...
2026-06-09T04:31:04.613107212Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T04:31:04.637336275Z tar: Removing leading `/' from member names
2026-06-09T04:31:04.646106830Z INFO: HAProxy config collection complete.
2026-06-09T04:31:04.668262565Z Wrote inspect data to must-gather.
2026-06-09T04:31:04.673665635Z No resources found
2026-06-09T04:31:04.702963551Z Wrote inspect data to must-gather.
2026-06-09T04:31:04.763538092Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-09T04:31:04.767316466Z Wrote inspect data to must-gather.
2026-06-09T04:31:04.772039582Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T04:31:04.788522890Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-09T04:31:04.891415625Z Wrote inspect data to must-gather.
2026-06-09T04:31:04.898324839Z Wrote inspect data to must-gather.
2026-06-09T04:31:04.935719930Z INFO: 'previous-logs' folder not found on ip-10-0-129-107.ec2.internal, skipping...
2026-06-09T04:31:05.098871849Z error: the server doesn't have a resource type "machineconfigs"
2026-06-09T04:31:05.211398521Z Wrote inspect data to must-gather.
2026-06-09T04:31:05.243226964Z INFO: 'previous-logs' folder not found on ip-10-0-135-201.ec2.internal, skipping...
2026-06-09T04:31:05.252589831Z INFO: Getting rules from prometheus-k8s-0
2026-06-09T04:31:05.264466444Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-09T04:31:05.293787339Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-09T04:31:05.492632004Z Wrote inspect data to must-gather.
2026-06-09T04:31:05.518166237Z INFO: 'previous-logs' folder not found on ip-10-0-136-47.ec2.internal, skipping...
2026-06-09T04:31:05.518203471Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-09T04:31:05.518217700Z INFO: Machine Config Daemon termination log collection complete.
2026-06-09T04:31:05.533674999Z Wrote inspect data to must-gather.
2026-06-09T04:31:05.684107412Z INFO: Getting status/config from prometheus-k8s-0
2026-06-09T04:31:05.721008821Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-09T04:31:05.760409234Z Wrote inspect data to must-gather.
2026-06-09T04:31:05.960113528Z Wrote inspect data to must-gather.
2026-06-09T04:31:05.975842690Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T04:31:06.094094899Z INFO: INTERCONNECT MODE
2026-06-09T04:31:06.094151484Z INFO: Gathering ovn-kubernetes DBs
2026-06-09T04:31:06.124596194Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-09T04:31:06.188324882Z Wrote inspect data to must-gather.
2026-06-09T04:31:06.293065889Z INFO: Gathering OVN_Northbound from ovnkube-node-82vrh...
2026-06-09T04:31:06.293345511Z INFO: Gathering OVN_Northbound from ovnkube-node-qks4p...
2026-06-09T04:31:06.293602783Z INFO: Gathering OVN_Northbound from ovnkube-node-sk92z...
2026-06-09T04:31:06.293886060Z INFO: Gathering OVN_Southbound from ovnkube-node-82vrh...
2026-06-09T04:31:06.294215214Z INFO: Gathering OVN_Southbound from ovnkube-node-qks4p...
2026-06-09T04:31:06.294515343Z INFO: Gathering OVN_Southbound from ovnkube-node-sk92z...
2026-06-09T04:31:06.344467784Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-09T04:31:06.382100316Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-09T04:31:06.572988271Z Wrote inspect data to must-gather.
2026-06-09T04:31:06.580542096Z tar: Removing leading `/' from member names
2026-06-09T04:31:06.621346816Z tar: Removing leading `/' from member names
2026-06-09T04:31:06.641732016Z tar: Removing leading `/' from member names
2026-06-09T04:31:06.657999738Z tar: Removing leading `/' from member names
2026-06-09T04:31:06.692301717Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:06.698944752Z tar: Removing leading `/' from member names
2026-06-09T04:31:06.710666488Z tar: Removing leading `/' from member names
2026-06-09T04:31:06.735605230Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-09T04:31:06.780005065Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T04:31:06.948529614Z Wrote inspect data to must-gather.
2026-06-09T04:31:06.974673889Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:06.977905820Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-09T04:31:07.018481170Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-09T04:31:07.025414074Z Gathering data for ns/openshift-network-console...
2026-06-09T04:31:07.217939989Z INFO: Waiting for network log collection to complete ...
2026-06-09T04:31:07.217965009Z INFO: Waiting for ovnk database copies to complete ...
2026-06-09T04:31:07.218357242Z INFO: Copying ovnk databases complete.
2026-06-09T04:31:07.218960672Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:07.220059314Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-06-09T04:31:07.222497944Z ovnk_database_store/
2026-06-09T04:31:07.222497944Z ovnk_database_store/ovnkube-node-82vrh_nbdb
2026-06-09T04:31:07.231343747Z ovnk_database_store/ovnkube-node-sk92z_sbdb
2026-06-09T04:31:07.249632555Z ovnk_database_store/ovnkube-node-sk92z_nbdb
2026-06-09T04:31:07.257488335Z ovnk_database_store/ovnkube-node-qks4p_sbdb
2026-06-09T04:31:07.274797086Z ovnk_database_store/ovnkube-node-qks4p_nbdb
2026-06-09T04:31:07.283283918Z ovnk_database_store/ovnkube-node-82vrh_sbdb
2026-06-09T04:31:07.301014607Z Wrote inspect data to must-gather.
2026-06-09T04:31:07.307252715Z INFO: Network log collection complete.
2026-06-09T04:31:07.442994441Z INFO: Getting status from alertmanager-main-0
2026-06-09T04:31:07.448780548Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:07.452265838Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-09T04:31:07.548480656Z Gathering data for ns/openshift-console-operator...
2026-06-09T04:31:07.640797452Z Wrote inspect data to must-gather.
2026-06-09T04:31:07.674308917Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:07.838423080Z W0609 04:31:07.838379    1556 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-09T04:31:07.873286712Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:07.876186367Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-09T04:31:07.935124520Z Wrote inspect data to must-gather.
2026-06-09T04:31:07.960885656Z Gathering data for ns/openshift-console...
2026-06-09T04:31:08.119797348Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:08.191471637Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-09T04:31:08.194429595Z INFO: Fallback to identify the container image from release info
2026-06-09T04:31:08.305065811Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:08.307431598Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-09T04:31:08.475645929Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:08.485917758Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-09T04:31:08.663284963Z Wrote inspect data to /must-gather/istio.
2026-06-09T04:31:08.921422960Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-09T04:31:08.994388176Z Gathering data for ns/openshift-dns-operator...
2026-06-09T04:31:09.071107351Z daemonset.apps/perf-node-gather-daemonset created
2026-06-09T04:31:09.203648128Z Waiting for performance profile collector pods to become ready: 1
2026-06-09T04:31:09.324304712Z Gathering data for ns/openshift-dns...
2026-06-09T04:31:09.379874179Z Done executing Istio gather script
2026-06-09T04:31:09.781031543Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T04:31:09.868508491Z Gathering data for ns/openshift-image-registry...
2026-06-09T04:31:10.297574147Z Gathering data for ns/openshift-ingress-operator...
2026-06-09T04:31:10.329699402Z Waiting for performance profile collector pods to become ready: 2
2026-06-09T04:31:10.600566036Z Gathering data for ns/openshift-ingress...
2026-06-09T04:31:10.896518570Z Gathering data for ns/openshift-ingress-canary...
2026-06-09T04:31:11.235147901Z Gathering data for ns/openshift-insights...
2026-06-09T04:31:11.440591693Z Waiting for performance profile collector pods to become ready: 3
2026-06-09T04:31:12.548877695Z Waiting for performance profile collector pods to become ready: 4
2026-06-09T04:31:13.003418940Z Gathering data for ns/openshift-jobset-operator...
2026-06-09T04:31:13.670385377Z Waiting for performance profile collector pods to become ready: 5
2026-06-09T04:31:13.772669417Z Gathering data for ns/openshift-config...
2026-06-09T04:31:14.009120986Z Gathering data for ns/openshift-config-managed...
2026-06-09T04:31:14.377243808Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-09T04:31:14.649283740Z Gathering data for ns/openshift-kube-apiserver...
2026-06-09T04:31:14.786933723Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-09T04:31:14.788934214Z Waiting for performance profile collector pods to become ready: 6
2026-06-09T04:31:14.903980574Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-09T04:31:15.200784168Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-09T04:31:15.453922566Z Gathering data for ns/openshift-kube-scheduler...
2026-06-09T04:31:15.686766562Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-09T04:31:15.911032259Z Waiting for performance profile collector pods to become ready: 7
2026-06-09T04:31:15.956257705Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-09T04:31:16.241853572Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-09T04:31:16.516706457Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-09T04:31:16.974245535Z Gathering data for ns/openshift-multus...
2026-06-09T04:31:17.037407821Z Waiting for performance profile collector pods to become ready: 8
2026-06-09T04:31:18.164668792Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-09T04:31:18.204490250Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-09T04:31:18.407678670Z Collecting performance related data for node ip-10-0-129-107.ec2.internal
2026-06-09T04:31:18.414273040Z Collecting performance related data for node ip-10-0-136-47.ec2.internal
2026-06-09T04:31:18.422212537Z Collecting performance related data for node ip-10-0-135-201.ec2.internal
2026-06-09T04:31:19.527661992Z Gathering data for ns/openshift-host-network...
2026-06-09T04:31:19.794675542Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-09T04:31:19.901336067Z Gathering data for ns/openshift-network-diagnostics...
2026-06-09T04:31:20.386709506Z Gathering data for ns/openshift-network-node-identity...
2026-06-09T04:31:20.792489169Z Gathering data for ns/openshift-network-operator...
2026-06-09T04:31:21.170924200Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-09T04:31:21.455419451Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-09T04:31:21.908538598Z Gathering data for ns/openshift-apiserver-operator...
2026-06-09T04:31:22.220620848Z Gathering data for ns/openshift-apiserver...
2026-06-09T04:31:22.593938665Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-09T04:31:22.844434865Z Gathering data for ns/openshift-controller-manager...
2026-06-09T04:31:23.110524048Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-09T04:31:23.773715073Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-09T04:31:24.049754549Z Gathering data for ns/openshift-service-ca-operator...
2026-06-09T04:31:24.339847755Z Gathering data for ns/openshift-service-ca...
2026-06-09T04:31:24.671744043Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-09T04:31:24.802489939Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-09T04:31:25.549062641Z Wrote inspect data to must-gather.
2026-06-09T04:31:25.549101873Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T04:31:25.549101873Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-09T04:31:26.306018309Z Collecting kubelet logs for node ip-10-0-129-107.ec2.internal
2026-06-09T04:31:26.307912710Z Collecting kubelet logs for node ip-10-0-136-47.ec2.internal
2026-06-09T04:31:26.309736667Z Collecting kubelet logs for node ip-10-0-135-201.ec2.internal
2026-06-09T04:31:26.609702923Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-mwhmf namespace
2026-06-09T04:31:26.614448770Z INFO: Node performance data collection complete.
2026-06-09T04:31:26.744285892Z error: the server doesn't have a resource type "clusters"
2026-06-09T04:31:26.884859964Z Caches written to disk