2026-06-11T20:58:47.767215041Z [disk usage checker] Started
2026-06-11T20:58:47.770741709Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T20:58:47.954269770Z Gathering data for ns/openshift-cluster-version...
2026-06-11T20:58:48.070409594Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-11T20:58:48.150746461Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T20:58:48.226088405Z Gathering data for ns/default...
2026-06-11T20:58:48.483639830Z Gathering data for ns/openshift...
2026-06-11T20:58:48.852049864Z Gathering data for ns/kube-system...
2026-06-11T20:58:49.457467849Z Gathering data for ns/openshift-etcd...
2026-06-11T20:58:49.720734746Z Wrote inspect data to must-gather.
2026-06-11T20:58:49.720776387Z error: inspection completed with the errors occurred while gathering data:
2026-06-11T20:58:49.720776387Z     namespaces "assisted-installer" not found
2026-06-11T20:58:50.278453438Z Waiting on subprocesses to finish execution.
2026-06-11T20:58:50.285258907Z INFO: Gathering HAProxy config files
2026-06-11T20:58:50.287263328Z Executing Istio gather script
2026-06-11T20:58:50.289136065Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-11T20:58:50.308480995Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-11T20:58:50.320308863Z INFO: Collecting host service logs for crio
2026-06-11T20:58:50.320620878Z INFO: Collecting host service logs for kubelet
2026-06-11T20:58:50.320934637Z INFO: Collecting host service logs for rpm-ostreed
2026-06-11T20:58:50.321226432Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-11T20:58:50.321570976Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-11T20:58:50.321862978Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-11T20:58:50.322157274Z INFO: Collecting host service logs for NetworkManager
2026-06-11T20:58:50.322473552Z INFO: Collecting host service logs for openvswitch
2026-06-11T20:58:50.322764824Z INFO: Collecting host service logs for ovs-configuration
2026-06-11T20:58:50.323063231Z INFO: Collecting host service logs for ovsdb-server
2026-06-11T20:58:50.323354565Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-11T20:58:50.323724157Z INFO: Waiting for worker host service log collection to complete ...
2026-06-11T20:58:50.329973813Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-11T20:58:50.542456324Z INFO: Waiting for node performance related collection to complete ...
2026-06-11T20:58:50.792783521Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-11T20:58:50.822930722Z INFO: Collecting Insights Archives from insights-operator-5bbd86d6bd-z825d
2026-06-11T20:58:50.822994977Z insights-runtime-extractor-cq7zt
2026-06-11T20:58:50.823023694Z insights-runtime-extractor-mr6g4
2026-06-11T20:58:50.823052135Z insights-runtime-extractor-zs95z
2026-06-11T20:58:50.992308692Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-11T20:58:50.992414374Z INFO: on-disk MachineConfig config collection complete.
2026-06-11T20:58:51.128499471Z No resources found in openshift-etcd namespace.
2026-06-11T20:58:51.336714063Z Wrote inspect data to must-gather.
2026-06-11T20:58:51.526206418Z INFO: "metallb-operator" not detected. Skipping.
2026-06-11T20:58:51.632096888Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-11T20:58:51.691485284Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-11T20:58:51.698647625Z error: the server doesn't have a resource type "performanceprofile"
2026-06-11T20:58:51.728022491Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-11T20:58:51.732495458Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-11T20:58:51.732495458Z See 'oc rsync -h' for help and examples
2026-06-11T20:58:51.810969461Z ERROR: No running kube-apiserver pods found
2026-06-11T20:58:51.858487949Z No resources found
2026-06-11T20:58:51.865152632Z INFO: Worker host service log collection to complete.
2026-06-11T20:58:51.870521266Z Wrote inspect data to must-gather.
2026-06-11T20:58:51.889167325Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-11T20:58:52.220474600Z Wrote inspect data to must-gather.
2026-06-11T20:58:52.256553906Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-11T20:58:52.288875096Z Gathering data for ns/openshift-monitoring...
2026-06-11T20:58:52.360751742Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T20:58:52.363708044Z INFO: 'previous-logs' folder not found on ip-10-0-132-179.ec2.internal, skipping...
2026-06-11T20:58:52.498210296Z INFO: HAProxy config collection complete.
2026-06-11T20:58:52.587999954Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-11T20:58:52.650959944Z No resources found
2026-06-11T20:58:52.656575359Z Wrote inspect data to must-gather.
2026-06-11T20:58:52.782162571Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T20:58:52.820918861Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-11T20:58:52.831727876Z error: the server doesn't have a resource type "machineconfigs"
2026-06-11T20:58:52.851949986Z Wrote inspect data to must-gather.
2026-06-11T20:58:52.881640088Z Wrote inspect data to must-gather.
2026-06-11T20:58:52.922557307Z Wrote inspect data to must-gather.
2026-06-11T20:58:52.966781018Z INFO: 'previous-logs' folder not found on ip-10-0-137-254.ec2.internal, skipping...
2026-06-11T20:58:53.013737200Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-11T20:58:53.130870730Z Wrote inspect data to must-gather.
2026-06-11T20:58:53.133304419Z INFO: Getting rules from prometheus-k8s-0
2026-06-11T20:58:53.251848761Z INFO: 'previous-logs' folder not found on ip-10-0-141-56.ec2.internal, skipping...
2026-06-11T20:58:53.251888370Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-11T20:58:53.251888370Z INFO: Machine Config Daemon termination log collection complete.
2026-06-11T20:58:53.258260306Z Wrote inspect data to must-gather.
2026-06-11T20:58:53.390218750Z Wrote inspect data to must-gather.
2026-06-11T20:58:53.408856326Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-11T20:58:53.536127519Z INFO: Getting status/config from prometheus-k8s-0
2026-06-11T20:58:53.630830016Z Wrote inspect data to must-gather.
2026-06-11T20:58:53.636012006Z Wrote inspect data to must-gather.
2026-06-11T20:58:53.789830750Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-11T20:58:53.835047663Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-11T20:58:53.851296368Z Wrote inspect data to must-gather.
2026-06-11T20:58:53.943751927Z INFO: INTERCONNECT MODE
2026-06-11T20:58:53.943798450Z INFO: Gathering ovn-kubernetes DBs
2026-06-11T20:58:53.967854602Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-11T20:58:54.023590058Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:54.099741352Z Wrote inspect data to must-gather.
2026-06-11T20:58:54.128667002Z INFO: Gathering OVN_Northbound from ovnkube-node-x5z7f...
2026-06-11T20:58:54.128912066Z INFO: Gathering OVN_Northbound from ovnkube-node-z8g6g...
2026-06-11T20:58:54.129110913Z INFO: Gathering OVN_Northbound from ovnkube-node-z9jtq...
2026-06-11T20:58:54.129388954Z INFO: Gathering OVN_Southbound from ovnkube-node-x5z7f...
2026-06-11T20:58:54.129631267Z INFO: Gathering OVN_Southbound from ovnkube-node-z8g6g...
2026-06-11T20:58:54.129900471Z INFO: Gathering OVN_Southbound from ovnkube-node-z9jtq...
2026-06-11T20:58:54.248594894Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-11T20:58:54.382777151Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:54.386818166Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-11T20:58:54.456900547Z tar: Removing leading `/' from member names
2026-06-11T20:58:54.489344208Z W0611 20:58:54.489296    1311 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-11T20:58:54.497137202Z Wrote inspect data to must-gather.
2026-06-11T20:58:54.502542697Z tar: Removing leading `/' from member names
2026-06-11T20:58:54.518353235Z tar: Removing leading `/' from member names
2026-06-11T20:58:54.533038501Z tar: Removing leading `/' from member names
2026-06-11T20:58:54.538497251Z tar: Removing leading `/' from member names
2026-06-11T20:58:54.583593288Z tar: Removing leading `/' from member names
2026-06-11T20:58:54.609766051Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T20:58:54.648981152Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-11T20:58:54.680986416Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:54.828382760Z Gathering data for ns/openshift-network-console...
2026-06-11T20:58:54.857966299Z Wrote inspect data to must-gather.
2026-06-11T20:58:54.919177898Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-11T20:58:54.946505685Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:54.952873803Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-11T20:58:55.009071461Z INFO: Waiting for network log collection to complete ...
2026-06-11T20:58:55.009106654Z INFO: Waiting for ovnk database copies to complete ...
2026-06-11T20:58:55.009510915Z INFO: Copying ovnk databases complete.
2026-06-11T20:58:55.011169303Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-06-11T20:58:55.013327526Z ovnk_database_store/
2026-06-11T20:58:55.013327526Z ovnk_database_store/ovnkube-node-x5z7f_sbdb
2026-06-11T20:58:55.042512528Z ovnk_database_store/ovnkube-node-z9jtq_sbdb
2026-06-11T20:58:55.067632309Z ovnk_database_store/ovnkube-node-x5z7f_nbdb
2026-06-11T20:58:55.082726804Z ovnk_database_store/ovnkube-node-z8g6g_sbdb
2026-06-11T20:58:55.105006631Z ovnk_database_store/ovnkube-node-z8g6g_nbdb
2026-06-11T20:58:55.116058533Z ovnk_database_store/ovnkube-node-z9jtq_nbdb
2026-06-11T20:58:55.128162179Z INFO: Network log collection complete.
2026-06-11T20:58:55.169040744Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:55.211839601Z Wrote inspect data to must-gather.
2026-06-11T20:58:55.303424686Z Gathering data for ns/openshift-console-operator...
2026-06-11T20:58:55.331723843Z INFO: Getting status from alertmanager-main-0
2026-06-11T20:58:55.414471021Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:55.418084901Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-11T20:58:55.525144451Z Wrote inspect data to must-gather.
2026-06-11T20:58:55.646219482Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:55.720577093Z Gathering data for ns/openshift-console...
2026-06-11T20:58:55.780206438Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-11T20:58:55.782386022Z INFO: Fallback to identify the container image from release info
2026-06-11T20:58:55.848648291Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:55.851973864Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-11T20:58:56.010386582Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:56.192239849Z Wrote inspect data to /must-gather/istio.
2026-06-11T20:58:56.207971985Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-11T20:58:56.379706276Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d66a6b2ec52dede5d83ce04f6abfc519df1fd7dc36248741e03ecb9bf8d1762
2026-06-11T20:58:56.514248078Z daemonset.apps/perf-node-gather-daemonset created
2026-06-11T20:58:56.649074887Z Waiting for performance profile collector pods to become ready: 1
2026-06-11T20:58:56.708338462Z Gathering data for ns/openshift-dns-operator...
2026-06-11T20:58:56.881432071Z Done executing Istio gather script
2026-06-11T20:58:56.986375271Z Gathering data for ns/openshift-dns...
2026-06-11T20:58:57.486500510Z Gathering data for ns/openshift-image-registry...
2026-06-11T20:58:57.766078604Z Waiting for performance profile collector pods to become ready: 2
2026-06-11T20:58:57.788754575Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T20:58:57.902795581Z Gathering data for ns/openshift-ingress-operator...
2026-06-11T20:58:58.177800364Z Gathering data for ns/openshift-ingress...
2026-06-11T20:58:58.452243904Z Gathering data for ns/openshift-ingress-canary...
2026-06-11T20:58:58.833129757Z Gathering data for ns/openshift-insights...
2026-06-11T20:58:58.882508583Z Waiting for performance profile collector pods to become ready: 3
2026-06-11T20:58:59.989016291Z Waiting for performance profile collector pods to become ready: 4
2026-06-11T20:59:00.687536432Z Gathering data for ns/openshift-config...
2026-06-11T20:59:00.911870674Z Gathering data for ns/openshift-config-managed...
2026-06-11T20:59:01.092294015Z Waiting for performance profile collector pods to become ready: 5
2026-06-11T20:59:01.265020759Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-11T20:59:01.478975358Z Gathering data for ns/openshift-kube-apiserver...
2026-06-11T20:59:01.693539520Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-11T20:59:01.938599386Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-11T20:59:02.182562611Z Gathering data for ns/openshift-kube-scheduler...
2026-06-11T20:59:02.218779575Z Waiting for performance profile collector pods to become ready: 6
2026-06-11T20:59:02.411123227Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-11T20:59:02.643566572Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-11T20:59:02.794306008Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T20:59:02.912569890Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-11T20:59:03.179721906Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-11T20:59:03.327219058Z Waiting for performance profile collector pods to become ready: 7
2026-06-11T20:59:03.617649685Z Gathering data for ns/openshift-multus...
2026-06-11T20:59:04.443761433Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-11T20:59:04.693297192Z Collecting performance related data for node ip-10-0-137-254.ec2.internal
2026-06-11T20:59:04.702694222Z Collecting performance related data for node ip-10-0-141-56.ec2.internal
2026-06-11T20:59:04.711050915Z Collecting performance related data for node ip-10-0-132-179.ec2.internal
2026-06-11T20:59:04.876570298Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-11T20:59:06.110377228Z Gathering data for ns/openshift-host-network...
2026-06-11T20:59:06.405703846Z Gathering data for ns/openshift-network-diagnostics...
2026-06-11T20:59:06.892328189Z Gathering data for ns/openshift-network-node-identity...
2026-06-11T20:59:07.304117252Z Gathering data for ns/openshift-network-operator...
2026-06-11T20:59:07.680189679Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-11T20:59:07.800812238Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T20:59:07.906229772Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-11T20:59:08.332542422Z Gathering data for ns/openshift-apiserver-operator...
2026-06-11T20:59:08.597365700Z Gathering data for ns/openshift-apiserver...
2026-06-11T20:59:08.960499003Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-11T20:59:09.201352824Z Gathering data for ns/openshift-controller-manager...
2026-06-11T20:59:09.465227544Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-11T20:59:10.080639869Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-11T20:59:10.347196015Z Gathering data for ns/openshift-service-ca-operator...
2026-06-11T20:59:10.611706934Z Gathering data for ns/openshift-service-ca...
2026-06-11T20:59:10.944431915Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-11T20:59:11.771043880Z Wrote inspect data to must-gather.
2026-06-11T20:59:11.771095749Z error: inspection completed with the errors occurred while gathering data:
2026-06-11T20:59:11.771095749Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-11T20:59:12.538994901Z Collecting kubelet logs for node ip-10-0-137-254.ec2.internal
2026-06-11T20:59:12.540675176Z Collecting kubelet logs for node ip-10-0-141-56.ec2.internal
2026-06-11T20:59:12.542359801Z Collecting kubelet logs for node ip-10-0-132-179.ec2.internal
2026-06-11T20:59:12.806424865Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-11T20:59:12.850277257Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-hskms namespace
2026-06-11T20:59:12.854888729Z INFO: Node performance data collection complete.
2026-06-11T20:59:12.975680623Z error: the server doesn't have a resource type "clusters"
2026-06-11T20:59:13.124727663Z Caches written to disk