2026-04-17T16:49:28.211611923Z [disk usage checker] Started
2026-04-17T16:49:28.242664301Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T16:49:28.444625880Z Gathering data for ns/openshift-cluster-version...
2026-04-17T16:49:28.612345635Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T16:49:28.714295693Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T16:49:28.817355483Z Gathering data for ns/default...
2026-04-17T16:49:29.183679867Z Gathering data for ns/openshift...
2026-04-17T16:49:29.658969290Z Gathering data for ns/kube-system...
2026-04-17T16:49:30.402814662Z Gathering data for ns/openshift-etcd...
2026-04-17T16:49:30.735483947Z Waiting on subprocesses to finish execution.
2026-04-17T16:49:30.742952291Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-17T16:49:30.748414392Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-17T16:49:30.749491738Z INFO: Gathering HAProxy config files
2026-04-17T16:49:30.754763217Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-17T16:49:30.774329716Z INFO: Collecting host service logs for crio
2026-04-17T16:49:30.774723368Z INFO: Collecting host service logs for kubelet
2026-04-17T16:49:30.775059740Z INFO: Collecting host service logs for rpm-ostreed
2026-04-17T16:49:30.775404905Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-17T16:49:30.775742367Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-17T16:49:30.776116695Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-17T16:49:30.776457595Z INFO: Collecting host service logs for NetworkManager
2026-04-17T16:49:30.776773516Z INFO: Collecting host service logs for openvswitch
2026-04-17T16:49:30.777064646Z INFO: Collecting host service logs for ovs-configuration
2026-04-17T16:49:30.777393791Z INFO: Collecting host service logs for ovsdb-server
2026-04-17T16:49:30.777687858Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-17T16:49:30.778041799Z INFO: Waiting for worker host service log collection to complete ...
2026-04-17T16:49:30.845834310Z Wrote inspect data to must-gather.
2026-04-17T16:49:30.845875608Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T16:49:30.845875608Z     namespaces "assisted-installer" not found
2026-04-17T16:49:30.901539286Z INFO: Waiting for node performance related collection to complete ...
2026-04-17T16:49:31.519620166Z No resources found
2026-04-17T16:49:31.582541786Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-mqbqx
2026-04-17T16:49:31.582594229Z insights-runtime-extractor-2dbmm
2026-04-17T16:49:31.582614913Z insights-runtime-extractor-gz7tf
2026-04-17T16:49:31.582633491Z insights-runtime-extractor-n6gcq
2026-04-17T16:49:31.773849452Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-17T16:49:31.798151884Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-17T16:49:31.891947925Z No resources found in openshift-etcd namespace.
2026-04-17T16:49:32.032336158Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-17T16:49:32.032364506Z INFO: on-disk MachineConfig config collection complete.
2026-04-17T16:49:32.075916692Z error: the server doesn't have a resource type "performanceprofile"
2026-04-17T16:49:32.107656995Z INFO: Worker host service log collection to complete.
2026-04-17T16:49:32.133286897Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-17T16:49:32.133286897Z See 'oc rsync -h' for help and examples
2026-04-17T16:49:32.189097308Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-17T16:49:32.227590911Z Wrote inspect data to must-gather.
2026-04-17T16:49:32.365573737Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-17T16:49:32.493423428Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-17T16:49:32.523346312Z ERROR: No running kube-apiserver pods found
2026-04-17T16:49:32.539303858Z INFO: "metallb-operator" not detected. Skipping.
2026-04-17T16:49:33.160971228Z No resources found
2026-04-17T16:49:33.211041479Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-17T16:49:33.223671592Z Wrote inspect data to must-gather.
2026-04-17T16:49:33.301950340Z INFO: HAProxy config collection complete.
2026-04-17T16:49:33.369848286Z Wrote inspect data to must-gather.
2026-04-17T16:49:33.397664247Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T16:49:33.414518893Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-17T16:49:33.462497011Z Gathering data for ns/kuadrant-system...
2026-04-17T16:49:33.497566418Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T16:49:33.516501012Z INFO: 'previous-logs' folder not found on ip-10-0-132-179.ec2.internal, skipping...
2026-04-17T16:49:33.572789395Z Wrote inspect data to must-gather.
2026-04-17T16:49:33.700428964Z Wrote inspect data to must-gather.
2026-04-17T16:49:33.709947091Z error: the server doesn't have a resource type "machineconfigs"
2026-04-17T16:49:33.731389945Z Wrote inspect data to must-gather.
2026-04-17T16:49:33.862715715Z Wrote inspect data to must-gather.
2026-04-17T16:49:33.914161290Z INFO: 'previous-logs' folder not found on ip-10-0-134-77.ec2.internal, skipping...
2026-04-17T16:49:34.065338254Z Wrote inspect data to must-gather.
2026-04-17T16:49:34.174219745Z INFO: Getting rules from prometheus-k8s-0
2026-04-17T16:49:34.324484957Z INFO: 'previous-logs' folder not found on ip-10-0-136-214.ec2.internal, skipping...
2026-04-17T16:49:34.324566885Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-17T16:49:34.324603053Z INFO: Machine Config Daemon termination log collection complete.
2026-04-17T16:49:34.335710736Z Wrote inspect data to must-gather.
2026-04-17T16:49:34.456210877Z Wrote inspect data to must-gather.
2026-04-17T16:49:34.616032819Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T16:49:34.680139425Z INFO: Getting status/config from prometheus-k8s-0
2026-04-17T16:49:34.687423922Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-17T16:49:34.715014244Z Wrote inspect data to must-gather.
2026-04-17T16:49:35.029725241Z Wrote inspect data to must-gather.
2026-04-17T16:49:35.133380317Z INFO: INTERCONNECT MODE
2026-04-17T16:49:35.133429315Z INFO: Gathering ovn-kubernetes DBs
2026-04-17T16:49:35.182448534Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T16:49:35.198108088Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-17T16:49:35.396235981Z INFO: Gathering OVN_Northbound from ovnkube-node-8qvmb...
2026-04-17T16:49:35.396602140Z INFO: Gathering OVN_Northbound from ovnkube-node-fpc4t...
2026-04-17T16:49:35.396899649Z INFO: Gathering OVN_Northbound from ovnkube-node-nhjqx...
2026-04-17T16:49:35.397246813Z INFO: Gathering OVN_Southbound from ovnkube-node-8qvmb...
2026-04-17T16:49:35.397863643Z INFO: Gathering OVN_Southbound from ovnkube-node-fpc4t...
2026-04-17T16:49:35.398189880Z INFO: Gathering OVN_Southbound from ovnkube-node-nhjqx...
2026-04-17T16:49:35.468051177Z Gathering data for ns/openshift-monitoring...
2026-04-17T16:49:35.513237172Z Wrote inspect data to must-gather.
2026-04-17T16:49:35.576456672Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-17T16:49:35.706180617Z tar: Removing leading `/' from member names
2026-04-17T16:49:35.728489288Z tar: Removing leading `/' from member names
2026-04-17T16:49:35.767045706Z tar: Removing leading `/' from member names
2026-04-17T16:49:35.778053342Z tar: Removing leading `/' from member names
2026-04-17T16:49:35.804102183Z tar: Removing leading `/' from member names
2026-04-17T16:49:35.892750166Z tar: Removing leading `/' from member names
2026-04-17T16:49:35.944458613Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-17T16:49:36.042547349Z Wrote inspect data to must-gather.
2026-04-17T16:49:36.245774378Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-17T16:49:36.435570049Z Wrote inspect data to must-gather.
2026-04-17T16:49:36.491430617Z INFO: Waiting for network log collection to complete ...
2026-04-17T16:49:36.491477982Z INFO: Waiting for ovnk database copies to complete ...
2026-04-17T16:49:36.491960573Z INFO: Copying ovnk databases complete.
2026-04-17T16:49:36.493712829Z 4.1M	must-gather/network_logs/ovnk_database_store
2026-04-17T16:49:36.495862457Z ovnk_database_store/
2026-04-17T16:49:36.495862457Z ovnk_database_store/ovnkube-node-nhjqx_sbdb
2026-04-17T16:49:36.517754567Z ovnk_database_store/ovnkube-node-8qvmb_sbdb
2026-04-17T16:49:36.544693214Z ovnk_database_store/ovnkube-node-8qvmb_nbdb
2026-04-17T16:49:36.557675684Z ovnk_database_store/ovnkube-node-fpc4t_sbdb
2026-04-17T16:49:36.585962374Z ovnk_database_store/ovnkube-node-fpc4t_nbdb
2026-04-17T16:49:36.600105204Z ovnk_database_store/ovnkube-node-nhjqx_nbdb
2026-04-17T16:49:36.612192191Z INFO: Network log collection complete.
2026-04-17T16:49:36.712001113Z INFO: Getting status from alertmanager-main-0
2026-04-17T16:49:36.813332658Z Wrote inspect data to must-gather.
2026-04-17T16:49:37.150195882Z Wrote inspect data to must-gather.
2026-04-17T16:49:37.469928467Z Wrote inspect data to must-gather.
2026-04-17T16:49:37.741308537Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-17T16:49:37.743583596Z INFO: Fallback to identify the container image from release info
2026-04-17T16:49:37.871840546Z Gathering data for ns/openshift-network-console...
2026-04-17T16:49:38.292143632Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-17T16:49:38.410186424Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T16:49:38.449323146Z daemonset.apps/perf-node-gather-daemonset created
2026-04-17T16:49:38.516277137Z Gathering data for ns/openshift-console-operator...
2026-04-17T16:49:38.625230327Z Waiting for performance profile collector pods to become ready: 1
2026-04-17T16:49:39.030008986Z Gathering data for ns/openshift-console...
2026-04-17T16:49:39.542529636Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-17T16:49:39.765462905Z Waiting for performance profile collector pods to become ready: 2
2026-04-17T16:49:39.974250078Z Gathering data for ns/openshift-dns-operator...
2026-04-17T16:49:40.131752770Z Wrote inspect data to must-gather.
2026-04-17T16:49:40.304054868Z Gathering data for ns/openshift-dns...
2026-04-17T16:49:40.894010402Z Waiting for performance profile collector pods to become ready: 3
2026-04-17T16:49:40.946546923Z Gathering data for ns/openshift-image-registry...
2026-04-17T16:49:41.445965710Z Gathering data for ns/openshift-ingress-operator...
2026-04-17T16:49:41.855616891Z Gathering data for ns/openshift-ingress...
2026-04-17T16:49:42.019832395Z Waiting for performance profile collector pods to become ready: 4
2026-04-17T16:49:42.553390193Z Gathering data for ns/openshift-ingress-canary...
2026-04-17T16:49:43.005312412Z Gathering data for ns/openshift-insights...
2026-04-17T16:49:43.136975543Z Waiting for performance profile collector pods to become ready: 5
2026-04-17T16:49:43.417612722Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T16:49:44.243742087Z Waiting for performance profile collector pods to become ready: 6
2026-04-17T16:49:45.049502531Z Gathering data for ns/opendatahub...
2026-04-17T16:49:45.362416877Z Waiting for performance profile collector pods to become ready: 7
2026-04-17T16:49:46.426538164Z Gathering data for ns/openshift-lws-operator...
2026-04-17T16:49:46.476389852Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-17T16:49:46.741545483Z Collecting performance related data for node ip-10-0-134-77.ec2.internal
2026-04-17T16:49:46.749456121Z Collecting performance related data for node ip-10-0-136-214.ec2.internal
2026-04-17T16:49:46.756701530Z Collecting performance related data for node ip-10-0-132-179.ec2.internal
2026-04-17T16:49:47.947955130Z Gathering data for ns/openshift-config...
2026-04-17T16:49:48.359922139Z Gathering data for ns/openshift-config-managed...
2026-04-17T16:49:48.423648938Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T16:49:48.863666009Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-17T16:49:49.182966099Z Gathering data for ns/openshift-kube-apiserver...
2026-04-17T16:49:49.531636901Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-17T16:49:49.894047292Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-17T16:49:50.274590687Z Gathering data for ns/openshift-kube-scheduler...
2026-04-17T16:49:50.584939153Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-17T16:49:50.942923988Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-17T16:49:51.357211936Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-17T16:49:51.724120791Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-17T16:49:52.316489553Z Gathering data for ns/openshift-multus...
2026-04-17T16:49:53.430691312Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T16:49:53.780118269Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-17T16:49:55.257320057Z Collecting kubelet logs for node ip-10-0-134-77.ec2.internal
2026-04-17T16:49:55.259005718Z Collecting kubelet logs for node ip-10-0-136-214.ec2.internal
2026-04-17T16:49:55.260773425Z Collecting kubelet logs for node ip-10-0-132-179.ec2.internal
2026-04-17T16:49:55.342286661Z Gathering data for ns/openshift-host-network...
2026-04-17T16:49:55.644591453Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-17T16:49:55.650227375Z INFO: Node performance data collection complete.
2026-04-17T16:49:55.779860632Z Gathering data for ns/openshift-network-diagnostics...
2026-04-17T16:49:56.391243545Z Gathering data for ns/openshift-network-node-identity...
2026-04-17T16:49:56.892994002Z Gathering data for ns/openshift-network-operator...
2026-04-17T16:49:57.305504877Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-17T16:49:57.626360658Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-17T16:49:58.288148614Z Gathering data for ns/openshift-apiserver-operator...
2026-04-17T16:49:58.435816537Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T16:49:58.641386693Z Gathering data for ns/openshift-apiserver...
2026-04-17T16:49:59.085053623Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-17T16:49:59.401646936Z Gathering data for ns/openshift-controller-manager...
2026-04-17T16:49:59.732824020Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-17T16:50:00.419941480Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-17T16:50:00.770605518Z Gathering data for ns/openshift-service-ca-operator...
2026-04-17T16:50:01.121485585Z Gathering data for ns/openshift-service-ca...
2026-04-17T16:50:01.527254844Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-17T16:50:02.577239339Z Wrote inspect data to must-gather.
2026-04-17T16:50:02.577289213Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T16:50:02.577289213Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-17T16:50:02.716516678Z error: the server doesn't have a resource type "clusters"
2026-04-17T16:50:02.895312796Z Caches written to disk