2026-04-24T14:48:42.026792223Z [disk usage checker] Started
2026-04-24T14:48:42.055305860Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-24T14:48:42.240800554Z Gathering data for ns/openshift-cluster-version...
2026-04-24T14:48:42.406674340Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T14:48:42.508162044Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T14:48:42.604653173Z Gathering data for ns/default...
2026-04-24T14:48:42.959508695Z Gathering data for ns/openshift...
2026-04-24T14:48:43.432044089Z Gathering data for ns/kube-system...
2026-04-24T14:48:44.225097943Z Gathering data for ns/openshift-etcd...
2026-04-24T14:48:44.470882353Z Waiting on subprocesses to finish execution.
2026-04-24T14:48:44.476646053Z INFO: Gathering HAProxy config files
2026-04-24T14:48:44.485891862Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T14:48:44.489175922Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T14:48:44.499956066Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T14:48:44.507964607Z INFO: Collecting host service logs for crio
2026-04-24T14:48:44.508390175Z INFO: Collecting host service logs for kubelet
2026-04-24T14:48:44.508753985Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T14:48:44.509156493Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T14:48:44.509500221Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T14:48:44.509853899Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T14:48:44.510196110Z INFO: Collecting host service logs for NetworkManager
2026-04-24T14:48:44.510518671Z INFO: Collecting host service logs for openvswitch
2026-04-24T14:48:44.510840542Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T14:48:44.511254063Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T14:48:44.511739818Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T14:48:44.512218270Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T14:48:44.635958479Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T14:48:44.828708707Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-rkjfd
2026-04-24T14:48:44.828765298Z insights-runtime-extractor-hp5nm
2026-04-24T14:48:44.828787411Z insights-runtime-extractor-q2c42
2026-04-24T14:48:44.828806934Z insights-runtime-extractor-xkrr7
2026-04-24T14:48:45.150973027Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T14:48:45.151023333Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T14:48:45.154485654Z Wrote inspect data to must-gather.
2026-04-24T14:48:45.154561581Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T14:48:45.154561581Z     namespaces "assisted-installer" not found
2026-04-24T14:48:45.247184752Z Wrote inspect data to must-gather.
2026-04-24T14:48:45.520624198Z No resources found
2026-04-24T14:48:45.529109319Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T14:48:45.598594017Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T14:48:45.652458061Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T14:48:45.822301783Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T14:48:45.834405210Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T14:48:45.928504803Z No resources found in openshift-etcd namespace.
2026-04-24T14:48:46.026723725Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T14:48:46.045310563Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T14:48:46.045310563Z See 'oc rsync -h' for help and examples
2026-04-24T14:48:46.089584624Z Wrote inspect data to must-gather.
2026-04-24T14:48:46.124443689Z INFO: Worker host service log collection to complete.
2026-04-24T14:48:46.148625046Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T14:48:46.212405750Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T14:48:46.496478175Z Wrote inspect data to must-gather.
2026-04-24T14:48:46.508505242Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T14:48:46.592298166Z Wrote inspect data to must-gather.
2026-04-24T14:48:46.654340707Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T14:48:46.687857834Z Wrote inspect data to must-gather.
2026-04-24T14:48:46.787530153Z No resources found
2026-04-24T14:48:46.857840358Z ERROR: No running kube-apiserver pods found
2026-04-24T14:48:46.876716824Z Wrote inspect data to must-gather.
2026-04-24T14:48:46.880457912Z Wrote inspect data to must-gather.
2026-04-24T14:48:46.906746041Z INFO: 'previous-logs' folder not found on ip-10-0-134-82.ec2.internal, skipping...
2026-04-24T14:48:46.942153685Z tar: Removing leading `/' from member names
2026-04-24T14:48:46.951013851Z INFO: HAProxy config collection complete.
2026-04-24T14:48:46.984286846Z Gathering data for ns/openshift-monitoring...
2026-04-24T14:48:47.066243135Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-24T14:48:47.072888808Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T14:48:47.154442009Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T14:48:47.159958645Z Wrote inspect data to must-gather.
2026-04-24T14:48:47.195815696Z INFO: 'previous-logs' folder not found on ip-10-0-137-95.ec2.internal, skipping...
2026-04-24T14:48:47.311766485Z Wrote inspect data to must-gather.
2026-04-24T14:48:47.421244432Z Wrote inspect data to must-gather.
2026-04-24T14:48:47.470442283Z INFO: 'previous-logs' folder not found on ip-10-0-143-92.ec2.internal, skipping...
2026-04-24T14:48:47.470484528Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T14:48:47.470516656Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T14:48:47.554976082Z Wrote inspect data to must-gather.
2026-04-24T14:48:47.581416668Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T14:48:47.669857006Z Wrote inspect data to must-gather.
2026-04-24T14:48:47.716560189Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T14:48:47.937779506Z Wrote inspect data to must-gather.
2026-04-24T14:48:47.962298160Z INFO: INTERCONNECT MODE
2026-04-24T14:48:47.962331166Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T14:48:48.018535269Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T14:48:48.133857482Z INFO: Gathering OVN_Northbound from ovnkube-node-7m9zt...
2026-04-24T14:48:48.134103756Z INFO: Gathering OVN_Northbound from ovnkube-node-hb7mf...
2026-04-24T14:48:48.134307281Z INFO: Gathering OVN_Northbound from ovnkube-node-tdc2t...
2026-04-24T14:48:48.134582051Z INFO: Gathering OVN_Southbound from ovnkube-node-7m9zt...
2026-04-24T14:48:48.134827355Z INFO: Gathering OVN_Southbound from ovnkube-node-hb7mf...
2026-04-24T14:48:48.135117160Z INFO: Gathering OVN_Southbound from ovnkube-node-tdc2t...
2026-04-24T14:48:48.153717235Z Wrote inspect data to must-gather.
2026-04-24T14:48:48.293162031Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T14:48:48.427537848Z tar: Removing leading `/' from member names
2026-04-24T14:48:48.431787770Z tar: Removing leading `/' from member names
2026-04-24T14:48:48.431866708Z tar: Removing leading `/' from member names
2026-04-24T14:48:48.436250717Z tar: Removing leading `/' from member names
2026-04-24T14:48:48.469739080Z tar: Removing leading `/' from member names
2026-04-24T14:48:48.473203898Z tar: Removing leading `/' from member names
2026-04-24T14:48:48.511469657Z Wrote inspect data to must-gather.
2026-04-24T14:48:48.589682778Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T14:48:48.728546780Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T14:48:48.762205451Z Wrote inspect data to must-gather.
2026-04-24T14:48:48.859298168Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T14:48:48.978561576Z INFO: Waiting for network log collection to complete ...
2026-04-24T14:48:48.978680183Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T14:48:48.979238234Z INFO: Copying ovnk databases complete.
2026-04-24T14:48:48.981589127Z 3.3M	must-gather/network_logs/ovnk_database_store
2026-04-24T14:48:48.984509540Z ovnk_database_store/
2026-04-24T14:48:48.984509540Z ovnk_database_store/ovnkube-node-7m9zt_nbdb
2026-04-24T14:48:48.995162918Z ovnk_database_store/ovnkube-node-hb7mf_nbdb
2026-04-24T14:48:49.004137921Z Wrote inspect data to must-gather.
2026-04-24T14:48:49.005673033Z ovnk_database_store/ovnkube-node-hb7mf_sbdb
2026-04-24T14:48:49.028995301Z ovnk_database_store/ovnkube-node-tdc2t_nbdb
2026-04-24T14:48:49.038241421Z ovnk_database_store/ovnkube-node-7m9zt_sbdb
2026-04-24T14:48:49.061890780Z ovnk_database_store/ovnkube-node-tdc2t_sbdb
2026-04-24T14:48:49.080437021Z INFO: Network log collection complete.
2026-04-24T14:48:49.205927300Z Wrote inspect data to must-gather.
2026-04-24T14:48:49.258941905Z INFO: Getting status from alertmanager-main-0
2026-04-24T14:48:49.327838226Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T14:48:49.500291304Z Gathering data for ns/openshift-network-console...
2026-04-24T14:48:49.501347251Z Wrote inspect data to must-gather.
2026-04-24T14:48:49.758689373Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T14:48:49.762241210Z INFO: Fallback to identify the container image from release info
2026-04-24T14:48:49.949338243Z Gathering data for ns/openshift-console-operator...
2026-04-24T14:48:50.331101481Z Gathering data for ns/openshift-console...
2026-04-24T14:48:50.396342924Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T14:48:50.533859744Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T14:48:50.665014724Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T14:48:50.853679904Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T14:48:51.268210007Z Gathering data for ns/openshift-dns-operator...
2026-04-24T14:48:51.609325488Z Gathering data for ns/openshift-dns...
2026-04-24T14:48:51.781032534Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T14:48:52.075477921Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-24T14:48:52.251464593Z Gathering data for ns/openshift-image-registry...
2026-04-24T14:48:52.772543944Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T14:48:52.910300172Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T14:48:53.149812029Z Gathering data for ns/openshift-ingress...
2026-04-24T14:48:53.529018731Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T14:48:53.959224685Z Gathering data for ns/openshift-insights...
2026-04-24T14:48:54.046256465Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T14:48:55.167158745Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T14:48:56.111409863Z Gathering data for ns/kserve...
2026-04-24T14:48:56.284322429Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T14:48:57.081734660Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-24T14:48:57.424657683Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T14:48:57.769489044Z Gathering data for ns/openshift-config...
2026-04-24T14:48:58.050306608Z Gathering data for ns/openshift-config-managed...
2026-04-24T14:48:58.540703680Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T14:48:58.558184131Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T14:48:58.832858597Z Collecting performance related data for node ip-10-0-143-92.ec2.internal
2026-04-24T14:48:58.841826885Z Collecting performance related data for node ip-10-0-134-82.ec2.internal
2026-04-24T14:48:58.848826962Z Collecting performance related data for node ip-10-0-137-95.ec2.internal
2026-04-24T14:48:58.869624044Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T14:48:59.256185382Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T14:48:59.621207789Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T14:49:00.042752692Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T14:49:00.400953617Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T14:49:00.760078417Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T14:49:01.120943468Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T14:49:01.501430036Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T14:49:02.088340282Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-24T14:49:02.110242925Z Gathering data for ns/openshift-multus...
2026-04-24T14:49:03.758872669Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T14:49:05.638129553Z Gathering data for ns/openshift-host-network...
2026-04-24T14:49:06.028834441Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T14:49:06.680096287Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T14:49:07.093726970Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-24T14:49:07.122034624Z Gathering data for ns/openshift-network-operator...
2026-04-24T14:49:07.253050404Z Collecting kubelet logs for node ip-10-0-143-92.ec2.internal
2026-04-24T14:49:07.255945605Z Collecting kubelet logs for node ip-10-0-134-82.ec2.internal
2026-04-24T14:49:07.258306327Z Collecting kubelet logs for node ip-10-0-137-95.ec2.internal
2026-04-24T14:49:07.578846236Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T14:49:07.632764862Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T14:49:07.639477270Z INFO: Node performance data collection complete.
2026-04-24T14:49:07.917090634Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T14:49:08.440136115Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T14:49:08.789453600Z Gathering data for ns/openshift-apiserver...
2026-04-24T14:49:09.199200447Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T14:49:09.483765678Z Gathering data for ns/openshift-controller-manager...
2026-04-24T14:49:09.780254685Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T14:49:10.478129978Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T14:49:10.776120335Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T14:49:11.113773301Z Gathering data for ns/openshift-service-ca...
2026-04-24T14:49:11.493055039Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T14:49:12.101013203Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-24T14:49:12.455810853Z Wrote inspect data to must-gather.
2026-04-24T14:49:12.455857571Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T14:49:12.455857571Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T14:49:12.592431614Z error: the server doesn't have a resource type "clusters"
2026-04-24T14:49:12.736478011Z Caches written to disk