2026-04-23T17:37:34.461866747Z [disk usage checker] Started
2026-04-23T17:37:34.465126363Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-04-23T17:37:34.890521468Z Gathering data for ns/openshift-cluster-version...
2026-04-23T17:37:35.039384879Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-23T17:37:35.139020588Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T17:37:35.225898094Z Gathering data for ns/default...
2026-04-23T17:37:35.547667792Z Gathering data for ns/openshift...
2026-04-23T17:37:35.982803307Z Gathering data for ns/kube-system...
2026-04-23T17:37:36.793025495Z Gathering data for ns/openshift-etcd...
2026-04-23T17:37:37.104039054Z Wrote inspect data to must-gather.
2026-04-23T17:37:37.104072990Z error: inspection completed with the errors occurred while gathering data:
2026-04-23T17:37:37.104072990Z     namespaces "assisted-installer" not found
2026-04-23T17:37:37.195733815Z Waiting on subprocesses to finish execution.
2026-04-23T17:37:37.204052032Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-23T17:37:37.206725936Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-23T17:37:37.211218835Z INFO: Gathering HAProxy config files
2026-04-23T17:37:37.213989819Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-23T17:37:37.245606002Z INFO: Waiting for node performance related collection to complete ...
2026-04-23T17:37:37.303293509Z INFO: Collecting host service logs for crio
2026-04-23T17:37:37.303648702Z INFO: Collecting host service logs for kubelet
2026-04-23T17:37:37.303996568Z INFO: Collecting host service logs for rpm-ostreed
2026-04-23T17:37:37.304310011Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-23T17:37:37.304630503Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-23T17:37:37.304964276Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-23T17:37:37.305277912Z INFO: Collecting host service logs for NetworkManager
2026-04-23T17:37:37.305587061Z INFO: Collecting host service logs for openvswitch
2026-04-23T17:37:37.305942614Z INFO: Collecting host service logs for ovs-configuration
2026-04-23T17:37:37.306261817Z INFO: Collecting host service logs for ovsdb-server
2026-04-23T17:37:37.306593404Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-23T17:37:37.306988625Z INFO: Waiting for worker host service log collection to complete ...
2026-04-23T17:37:37.594459570Z No resources found in openshift-etcd namespace.
2026-04-23T17:37:37.703979121Z INFO: "metallb-operator" not detected. Skipping.
2026-04-23T17:37:37.961754718Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-23T17:37:37.974527909Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-23T17:37:37.979688792Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-23T17:37:38.162363471Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-23T17:37:38.382842269Z No resources found
2026-04-23T17:37:38.481795619Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-sqtch
2026-04-23T17:37:38.481856297Z insights-runtime-extractor-56lmt
2026-04-23T17:37:38.481876362Z insights-runtime-extractor-f2c8s
2026-04-23T17:37:38.481896790Z insights-runtime-extractor-sbjdh
2026-04-23T17:37:38.575755633Z error: the server doesn't have a resource type "performanceprofile"
2026-04-23T17:37:38.754800963Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-23T17:37:38.754829218Z INFO: on-disk MachineConfig config collection complete.
2026-04-23T17:37:38.873469083Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-23T17:37:38.903373481Z INFO: Worker host service log collection to complete.
2026-04-23T17:37:38.920528327Z Wrote inspect data to must-gather.
2026-04-23T17:37:39.001823387Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-23T17:37:39.150385610Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T17:37:39.158054729Z ERROR: No running kube-apiserver pods found
2026-04-23T17:37:39.180247037Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-23T17:37:39.180247037Z See 'oc rsync -h' for help and examples
2026-04-23T17:37:39.331398662Z Wrote inspect data to must-gather.
2026-04-23T17:37:39.346942164Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-23T17:37:39.460925459Z Wrote inspect data to must-gather.
2026-04-23T17:37:39.473286788Z Wrote inspect data to must-gather.
2026-04-23T17:37:39.504155143Z INFO: 'previous-logs' folder not found on ip-10-0-130-110.ec2.internal, skipping...
2026-04-23T17:37:39.534155372Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-04-23T17:37:39.555364100Z Wrote inspect data to must-gather.
2026-04-23T17:37:39.564257620Z No resources found
2026-04-23T17:37:39.683986192Z tar: Removing leading `/' from member names
2026-04-23T17:37:39.694105948Z Wrote inspect data to must-gather.
2026-04-23T17:37:39.694914378Z INFO: HAProxy config collection complete.
2026-04-23T17:37:39.703863566Z Wrote inspect data to must-gather.
2026-04-23T17:37:39.830579079Z INFO: Getting rules from prometheus-k8s-0
2026-04-23T17:37:39.881360739Z Gathering data for ns/openshift-monitoring...
2026-04-23T17:37:39.890184049Z error: the server doesn't have a resource type "machineconfigs"
2026-04-23T17:37:39.914610697Z INFO: 'previous-logs' folder not found on ip-10-0-134-187.ec2.internal, skipping...
2026-04-23T17:37:39.998298717Z Wrote inspect data to must-gather.
2026-04-23T17:37:40.134725027Z Wrote inspect data to must-gather.
2026-04-23T17:37:40.186344736Z INFO: 'previous-logs' folder not found on ip-10-0-135-57.ec2.internal, skipping...
2026-04-23T17:37:40.186382198Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-23T17:37:40.186392625Z INFO: Machine Config Daemon termination log collection complete.
2026-04-23T17:37:40.256721531Z Wrote inspect data to must-gather.
2026-04-23T17:37:40.296868522Z INFO: Getting status/config from prometheus-k8s-0
2026-04-23T17:37:40.360957470Z Wrote inspect data to must-gather.
2026-04-23T17:37:40.496978568Z Wrote inspect data to must-gather.
2026-04-23T17:37:40.517801865Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-23T17:37:40.728885802Z Wrote inspect data to must-gather.
2026-04-23T17:37:40.773874518Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-23T17:37:40.782608431Z INFO: INTERCONNECT MODE
2026-04-23T17:37:40.782643093Z INFO: Gathering ovn-kubernetes DBs
2026-04-23T17:37:40.953432828Z Wrote inspect data to must-gather.
2026-04-23T17:37:40.956696013Z INFO: Gathering OVN_Northbound from ovnkube-node-75zl8...
2026-04-23T17:37:40.956938101Z INFO: Gathering OVN_Northbound from ovnkube-node-hc9pq...
2026-04-23T17:37:40.957158043Z INFO: Gathering OVN_Northbound from ovnkube-node-krt5k...
2026-04-23T17:37:40.957444557Z INFO: Gathering OVN_Southbound from ovnkube-node-75zl8...
2026-04-23T17:37:40.957698675Z INFO: Gathering OVN_Southbound from ovnkube-node-hc9pq...
2026-04-23T17:37:40.958037752Z INFO: Gathering OVN_Southbound from ovnkube-node-krt5k...
2026-04-23T17:37:41.103736775Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-23T17:37:41.264979989Z tar: Removing leading `/' from member names
2026-04-23T17:37:41.265638405Z tar: Removing leading `/' from member names
2026-04-23T17:37:41.273757853Z tar: Removing leading `/' from member names
2026-04-23T17:37:41.288279323Z tar: Removing leading `/' from member names
2026-04-23T17:37:41.288410527Z tar: Removing leading `/' from member names
2026-04-23T17:37:41.295655412Z tar: Removing leading `/' from member names
2026-04-23T17:37:41.311609094Z Wrote inspect data to must-gather.
2026-04-23T17:37:41.422245831Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-23T17:37:41.539997293Z Wrote inspect data to must-gather.
2026-04-23T17:37:41.714981790Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-23T17:37:41.776367458Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-23T17:37:41.779296210Z Wrote inspect data to must-gather.
2026-04-23T17:37:41.784231121Z INFO: Waiting for network log collection to complete ...
2026-04-23T17:37:41.784281422Z INFO: Waiting for ovnk database copies to complete ...
2026-04-23T17:37:41.784828193Z INFO: Copying ovnk databases complete.
2026-04-23T17:37:41.786800567Z 4.4M	must-gather/network_logs/ovnk_database_store
2026-04-23T17:37:41.788921695Z ovnk_database_store/
2026-04-23T17:37:41.788921695Z ovnk_database_store/ovnkube-node-hc9pq_sbdb
2026-04-23T17:37:41.813041822Z ovnk_database_store/ovnkube-node-75zl8_nbdb
2026-04-23T17:37:41.825400774Z ovnk_database_store/ovnkube-node-krt5k_sbdb
2026-04-23T17:37:41.852842093Z ovnk_database_store/ovnkube-node-hc9pq_nbdb
2026-04-23T17:37:41.863793550Z ovnk_database_store/ovnkube-node-krt5k_nbdb
2026-04-23T17:37:41.875139075Z ovnk_database_store/ovnkube-node-75zl8_sbdb
2026-04-23T17:37:41.898894311Z INFO: Network log collection complete.
2026-04-23T17:37:41.993219682Z Wrote inspect data to must-gather.
2026-04-23T17:37:42.128172547Z INFO: Getting status from alertmanager-main-0
2026-04-23T17:37:42.205481157Z Wrote inspect data to must-gather.
2026-04-23T17:37:42.334411575Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-23T17:37:42.472585998Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-23T17:37:42.475581694Z INFO: Fallback to identify the container image from release info
2026-04-23T17:37:42.500730958Z Gathering data for ns/openshift-network-console...
2026-04-23T17:37:42.901538216Z Gathering data for ns/openshift-console-operator...
2026-04-23T17:37:42.973450654Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-23T17:37:43.115113701Z daemonset.apps/perf-node-gather-daemonset created
2026-04-23T17:37:43.250900345Z Waiting for performance profile collector pods to become ready: 1
2026-04-23T17:37:43.313297459Z Gathering data for ns/openshift-console...
2026-04-23T17:37:43.757498117Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-23T17:37:44.131063127Z Gathering data for ns/openshift-dns-operator...
2026-04-23T17:37:44.388367942Z Waiting for performance profile collector pods to become ready: 2
2026-04-23T17:37:44.427184277Z Gathering data for ns/openshift-dns...
2026-04-23T17:37:44.541944637Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-04-23T17:37:45.035198748Z Gathering data for ns/openshift-image-registry...
2026-04-23T17:37:45.531240347Z Waiting for performance profile collector pods to become ready: 3
2026-04-23T17:37:45.611176109Z Gathering data for ns/openshift-ingress-operator...
2026-04-23T17:37:45.948630608Z Gathering data for ns/openshift-ingress...
2026-04-23T17:37:46.297356326Z Gathering data for ns/openshift-ingress-canary...
2026-04-23T17:37:46.679459146Z Waiting for performance profile collector pods to become ready: 4
2026-04-23T17:37:46.708492935Z Gathering data for ns/openshift-insights...
2026-04-23T17:37:47.800364779Z Waiting for performance profile collector pods to become ready: 5
2026-04-23T17:37:48.858517280Z Gathering data for ns/kserve...
2026-04-23T17:37:48.920074343Z Waiting for performance profile collector pods to become ready: 6
2026-04-23T17:37:49.547389338Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-04-23T17:37:50.070557608Z Waiting for performance profile collector pods to become ready: 7
2026-04-23T17:37:50.920829656Z Gathering data for ns/openshift-config...
2026-04-23T17:37:51.210092798Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-23T17:37:51.216561824Z Gathering data for ns/openshift-config-managed...
2026-04-23T17:37:51.487101476Z Collecting performance related data for node ip-10-0-130-110.ec2.internal
2026-04-23T17:37:51.494668827Z Collecting performance related data for node ip-10-0-135-57.ec2.internal
2026-04-23T17:37:51.502268564Z Collecting performance related data for node ip-10-0-134-187.ec2.internal
2026-04-23T17:37:51.754648979Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-23T17:37:52.111932391Z Gathering data for ns/openshift-kube-apiserver...
2026-04-23T17:37:52.476004572Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-23T17:37:52.865495906Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-23T17:37:53.252324597Z Gathering data for ns/openshift-kube-scheduler...
2026-04-23T17:37:53.581548335Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-23T17:37:53.897372415Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-23T17:37:54.299839310Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-23T17:37:54.554498442Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-04-23T17:37:54.659818287Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-23T17:37:55.225049031Z Gathering data for ns/openshift-multus...
2026-04-23T17:37:56.711096141Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-23T17:37:58.677612449Z Gathering data for ns/openshift-host-network...
2026-04-23T17:37:59.009175225Z Gathering data for ns/openshift-network-diagnostics...
2026-04-23T17:37:59.559732302Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-04-23T17:37:59.603567983Z Gathering data for ns/openshift-network-node-identity...
2026-04-23T17:38:00.019423933Z Gathering data for ns/openshift-network-operator...
2026-04-23T17:38:00.047599553Z Collecting kubelet logs for node ip-10-0-130-110.ec2.internal
2026-04-23T17:38:00.049371536Z Collecting kubelet logs for node ip-10-0-135-57.ec2.internal
2026-04-23T17:38:00.051164647Z Collecting kubelet logs for node ip-10-0-134-187.ec2.internal
2026-04-23T17:38:00.472379324Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-23T17:38:00.476207078Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-23T17:38:00.482473365Z INFO: Node performance data collection complete.
2026-04-23T17:38:00.825431649Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-23T17:38:01.285511817Z Gathering data for ns/openshift-apiserver-operator...
2026-04-23T17:38:01.635465931Z Gathering data for ns/openshift-apiserver...
2026-04-23T17:38:02.061622359Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-23T17:38:02.324299625Z Gathering data for ns/openshift-controller-manager...
2026-04-23T17:38:02.592115213Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-23T17:38:03.256450873Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-23T17:38:03.575645660Z Gathering data for ns/openshift-service-ca-operator...
2026-04-23T17:38:03.896909251Z Gathering data for ns/openshift-service-ca...
2026-04-23T17:38:04.296890948Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-23T17:38:04.566485276Z [disk usage checker] Volume usage percentage: current = 37 ; allowed = 70
2026-04-23T17:38:05.208101513Z Wrote inspect data to must-gather.
2026-04-23T17:38:05.208151487Z error: inspection completed with the errors occurred while gathering data:
2026-04-23T17:38:05.208151487Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-23T17:38:05.348679660Z error: the server doesn't have a resource type "clusters"
2026-04-23T17:38:05.509793826Z Caches written to disk