2026-04-21T05:00:36.270867882Z [disk usage checker] Started
2026-04-21T05:00:36.273981739Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70
2026-04-21T05:00:36.602101491Z Gathering data for ns/openshift-cluster-version...
2026-04-21T05:00:36.739045226Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-21T05:00:36.824695228Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-21T05:00:36.905442261Z Gathering data for ns/default...
2026-04-21T05:00:37.228912071Z Gathering data for ns/openshift...
2026-04-21T05:00:37.636115098Z Gathering data for ns/kube-system...
2026-04-21T05:00:38.363030387Z Gathering data for ns/openshift-etcd...
2026-04-21T05:00:38.701070976Z Wrote inspect data to must-gather.
2026-04-21T05:00:38.701189825Z error: inspection completed with the errors occurred while gathering data:
2026-04-21T05:00:38.701189825Z     namespaces "assisted-installer" not found
2026-04-21T05:00:38.823624571Z Waiting on subprocesses to finish execution.
2026-04-21T05:00:38.828888405Z INFO: Collecting host service logs for crio
2026-04-21T05:00:38.829204926Z INFO: Collecting host service logs for kubelet
2026-04-21T05:00:38.829530480Z INFO: Collecting host service logs for rpm-ostreed
2026-04-21T05:00:38.829873566Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-21T05:00:38.830223229Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-21T05:00:38.830560305Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-21T05:00:38.830936239Z INFO: Collecting host service logs for NetworkManager
2026-04-21T05:00:38.831312717Z INFO: Collecting host service logs for openvswitch
2026-04-21T05:00:38.831740523Z INFO: Collecting host service logs for ovs-configuration
2026-04-21T05:00:38.832094780Z INFO: Collecting host service logs for ovsdb-server
2026-04-21T05:00:38.832201425Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-21T05:00:38.832428428Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-21T05:00:38.832796562Z INFO: Waiting for worker host service log collection to complete ...
2026-04-21T05:00:38.844555507Z INFO: Gathering HAProxy config files
2026-04-21T05:00:38.847419901Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-21T05:00:38.852322092Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-21T05:00:39.062479239Z INFO: Waiting for node performance related collection to complete ...
2026-04-21T05:00:39.248992267Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-21T05:00:39.601076751Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-21T05:00:39.642461339Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-s5w6b
2026-04-21T05:00:39.642919849Z insights-runtime-extractor-4sndf
2026-04-21T05:00:39.642955544Z insights-runtime-extractor-jbmcx
2026-04-21T05:00:39.642977177Z insights-runtime-extractor-s9nkw
2026-04-21T05:00:39.735300623Z Wrote inspect data to must-gather.
2026-04-21T05:00:39.809602631Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-21T05:00:39.809837942Z INFO: "metallb-operator" not detected. Skipping.
2026-04-21T05:00:39.955004297Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-21T05:00:39.955036091Z INFO: on-disk MachineConfig config collection complete.
2026-04-21T05:00:39.983838795Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-21T05:00:40.070753227Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-21T05:00:40.148907826Z No resources found
2026-04-21T05:00:40.317252359Z No resources found in openshift-etcd namespace.
2026-04-21T05:00:40.445978355Z INFO: Worker host service log collection to complete.
2026-04-21T05:00:40.620840316Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-21T05:00:40.731342147Z Wrote inspect data to must-gather.
2026-04-21T05:00:40.756563967Z ERROR: No running kube-apiserver pods found
2026-04-21T05:00:40.766860081Z Wrote inspect data to must-gather.
2026-04-21T05:00:40.807111718Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-21T05:00:40.807111718Z See 'oc rsync -h' for help and examples
2026-04-21T05:00:40.809107668Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-21T05:00:40.824802956Z error: the server doesn't have a resource type "performanceprofile"
2026-04-21T05:00:40.856882843Z No resources found
2026-04-21T05:00:40.947714602Z tar: Removing leading `/' from member names
2026-04-21T05:00:40.953956545Z INFO: HAProxy config collection complete.
2026-04-21T05:00:41.007020280Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-21T05:00:41.058849302Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.074536018Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.119145387Z INFO: 'previous-logs' folder not found on ip-10-0-128-88.ec2.internal, skipping...
2026-04-21T05:00:41.166359846Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.205600478Z Gathering data for ns/openshift-monitoring...
2026-04-21T05:00:41.206676442Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.280938249Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70
2026-04-21T05:00:41.376166605Z INFO: Getting rules from prometheus-k8s-0
2026-04-21T05:00:41.387728492Z error: the server doesn't have a resource type "machineconfigs"
2026-04-21T05:00:41.405093488Z INFO: 'previous-logs' folder not found on ip-10-0-131-182.ec2.internal, skipping...
2026-04-21T05:00:41.449732413Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.623242781Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.677404755Z INFO: 'previous-logs' folder not found on ip-10-0-134-15.ec2.internal, skipping...
2026-04-21T05:00:41.677444527Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-21T05:00:41.677455194Z INFO: Machine Config Daemon termination log collection complete.
2026-04-21T05:00:41.720210862Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.788343750Z INFO: Getting status/config from prometheus-k8s-0
2026-04-21T05:00:41.837635435Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.955959733Z Wrote inspect data to must-gather.
2026-04-21T05:00:41.986814037Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-21T05:00:42.196117228Z Wrote inspect data to must-gather.
2026-04-21T05:00:42.210187601Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-21T05:00:42.245860639Z INFO: INTERCONNECT MODE
2026-04-21T05:00:42.245899667Z INFO: Gathering ovn-kubernetes DBs
2026-04-21T05:00:42.430899925Z INFO: Gathering OVN_Northbound from ovnkube-node-9ssvt...
2026-04-21T05:00:42.431110481Z INFO: Gathering OVN_Northbound from ovnkube-node-gl9hq...
2026-04-21T05:00:42.431334838Z INFO: Gathering OVN_Northbound from ovnkube-node-kxcbd...
2026-04-21T05:00:42.431629380Z INFO: Gathering OVN_Southbound from ovnkube-node-9ssvt...
2026-04-21T05:00:42.431971700Z INFO: Gathering OVN_Southbound from ovnkube-node-gl9hq...
2026-04-21T05:00:42.432213819Z INFO: Gathering OVN_Southbound from ovnkube-node-kxcbd...
2026-04-21T05:00:42.441739939Z Wrote inspect data to must-gather.
2026-04-21T05:00:42.479720661Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-21T05:00:42.584897130Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-21T05:00:42.724611061Z tar: Removing leading `/' from member names
2026-04-21T05:00:42.733705700Z tar: Removing leading `/' from member names
2026-04-21T05:00:42.735871356Z tar: Removing leading `/' from member names
2026-04-21T05:00:42.748494579Z tar: Removing leading `/' from member names
2026-04-21T05:00:42.765044180Z tar: Removing leading `/' from member names
2026-04-21T05:00:42.766911287Z tar: Removing leading `/' from member names
2026-04-21T05:00:42.787025930Z Wrote inspect data to must-gather.
2026-04-21T05:00:42.800562304Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-21T05:00:43.029884661Z Wrote inspect data to must-gather.
2026-04-21T05:00:43.062398290Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-21T05:00:43.244606568Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-21T05:00:43.250665361Z INFO: Waiting for network log collection to complete ...
2026-04-21T05:00:43.250687864Z INFO: Waiting for ovnk database copies to complete ...
2026-04-21T05:00:43.251083380Z INFO: Copying ovnk databases complete.
2026-04-21T05:00:43.252541680Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-04-21T05:00:43.255048322Z ovnk_database_store/
2026-04-21T05:00:43.255048322Z ovnk_database_store/ovnkube-node-gl9hq_sbdb
2026-04-21T05:00:43.281228973Z ovnk_database_store/ovnkube-node-gl9hq_nbdb
2026-04-21T05:00:43.290127449Z ovnk_database_store/ovnkube-node-9ssvt_nbdb
2026-04-21T05:00:43.300843466Z ovnk_database_store/ovnkube-node-kxcbd_sbdb
2026-04-21T05:00:43.329339342Z ovnk_database_store/ovnkube-node-9ssvt_sbdb
2026-04-21T05:00:43.351480126Z ovnk_database_store/ovnkube-node-kxcbd_nbdb
2026-04-21T05:00:43.357715912Z Wrote inspect data to must-gather.
2026-04-21T05:00:43.364011907Z INFO: Network log collection complete.
2026-04-21T05:00:43.428779271Z Gathering data for ns/openshift-network-console...
2026-04-21T05:00:43.443920191Z INFO: Getting status from alertmanager-main-0
2026-04-21T05:00:43.647359636Z Wrote inspect data to must-gather.
2026-04-21T05:00:43.850311855Z Gathering data for ns/openshift-console-operator...
2026-04-21T05:00:43.934356228Z Wrote inspect data to must-gather.
2026-04-21T05:00:44.189924830Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-21T05:00:44.194856864Z INFO: Fallback to identify the container image from release info
2026-04-21T05:00:44.234732411Z Gathering data for ns/openshift-console...
2026-04-21T05:00:44.711184692Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-21T05:00:44.731522404Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-21T05:00:44.858062028Z daemonset.apps/perf-node-gather-daemonset created
2026-04-21T05:00:44.991651189Z Waiting for performance profile collector pods to become ready: 1
2026-04-21T05:00:45.083421362Z Gathering data for ns/openshift-dns-operator...
2026-04-21T05:00:45.374983468Z Gathering data for ns/openshift-dns...
2026-04-21T05:00:45.944867584Z Gathering data for ns/openshift-image-registry...
2026-04-21T05:00:46.119243829Z Waiting for performance profile collector pods to become ready: 2
2026-04-21T05:00:46.286779304Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70
2026-04-21T05:00:46.417876953Z Gathering data for ns/openshift-ingress-operator...
2026-04-21T05:00:46.735678566Z Gathering data for ns/openshift-ingress...
2026-04-21T05:00:47.070236729Z Gathering data for ns/openshift-ingress-canary...
2026-04-21T05:00:47.250130310Z Waiting for performance profile collector pods to become ready: 3
2026-04-21T05:00:47.457552328Z Gathering data for ns/openshift-insights...
2026-04-21T05:00:48.366681936Z Waiting for performance profile collector pods to become ready: 4
2026-04-21T05:00:49.490928129Z Waiting for performance profile collector pods to become ready: 5
2026-04-21T05:00:49.492891085Z Gathering data for ns/kserve...
2026-04-21T05:00:50.620918854Z Waiting for performance profile collector pods to become ready: 6
2026-04-21T05:00:51.238242720Z Gathering data for ns/openshift-config...
2026-04-21T05:00:51.295161772Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70
2026-04-21T05:00:51.502969041Z Gathering data for ns/openshift-config-managed...
2026-04-21T05:00:51.740354410Z Waiting for performance profile collector pods to become ready: 7
2026-04-21T05:00:51.964440125Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-21T05:00:52.227490171Z Gathering data for ns/openshift-kube-apiserver...
2026-04-21T05:00:52.520930282Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-21T05:00:52.799263830Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-21T05:00:52.879718044Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-21T05:00:53.113017598Z Gathering data for ns/openshift-kube-scheduler...
2026-04-21T05:00:53.149979273Z Collecting performance related data for node ip-10-0-134-15.ec2.internal
2026-04-21T05:00:53.160008210Z Collecting performance related data for node ip-10-0-131-182.ec2.internal
2026-04-21T05:00:53.166801899Z Collecting performance related data for node ip-10-0-128-88.ec2.internal
2026-04-21T05:00:53.408221290Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-21T05:00:53.790289594Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-21T05:00:54.169930520Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-21T05:00:54.548308963Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-21T05:00:55.076138744Z Gathering data for ns/openshift-multus...
2026-04-21T05:00:56.304110770Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70
2026-04-21T05:00:56.468791837Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-21T05:00:57.985990419Z Gathering data for ns/openshift-host-network...
2026-04-21T05:00:58.285419695Z Gathering data for ns/openshift-network-diagnostics...
2026-04-21T05:00:58.807448097Z Gathering data for ns/openshift-network-node-identity...
2026-04-21T05:00:59.216404202Z Gathering data for ns/openshift-network-operator...
2026-04-21T05:00:59.581134092Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-21T05:00:59.834204707Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-21T05:01:00.297518709Z Gathering data for ns/openshift-apiserver-operator...
2026-04-21T05:01:00.646293218Z Gathering data for ns/openshift-apiserver...
2026-04-21T05:01:01.040101792Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-21T05:01:01.308684094Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70
2026-04-21T05:01:01.335269874Z Gathering data for ns/openshift-controller-manager...
2026-04-21T05:01:01.352728397Z Collecting kubelet logs for node ip-10-0-134-15.ec2.internal
2026-04-21T05:01:01.354851940Z Collecting kubelet logs for node ip-10-0-131-182.ec2.internal
2026-04-21T05:01:01.356990730Z Collecting kubelet logs for node ip-10-0-128-88.ec2.internal
2026-04-21T05:01:01.688025566Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-21T05:01:01.795854646Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-21T05:01:01.803528659Z INFO: Node performance data collection complete.
2026-04-21T05:01:02.343197495Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-21T05:01:02.615305131Z Gathering data for ns/openshift-service-ca-operator...
2026-04-21T05:01:02.931142305Z Gathering data for ns/openshift-service-ca...
2026-04-21T05:01:03.291302357Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-21T05:01:04.133792514Z Wrote inspect data to must-gather.
2026-04-21T05:01:04.133831949Z error: inspection completed with the errors occurred while gathering data:
2026-04-21T05:01:04.133831949Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-21T05:01:04.261130266Z error: the server doesn't have a resource type "clusters"
2026-04-21T05:01:04.414786579Z Caches written to disk