2026-04-16T18:56:12.127161771Z [disk usage checker] Started
2026-04-16T18:56:12.130529702Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-16T18:56:12.331130392Z Gathering data for ns/openshift-cluster-version...
2026-04-16T18:56:12.461035959Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T18:56:12.610581911Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T18:56:12.696623995Z Gathering data for ns/default...
2026-04-16T18:56:13.015784536Z Gathering data for ns/openshift...
2026-04-16T18:56:13.433166476Z Gathering data for ns/kube-system...
2026-04-16T18:56:14.141794614Z Gathering data for ns/openshift-etcd...
2026-04-16T18:56:14.469566352Z Wrote inspect data to must-gather.
2026-04-16T18:56:14.469598967Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T18:56:14.469598967Z     namespaces "assisted-installer" not found
2026-04-16T18:56:14.495020284Z Waiting on subprocesses to finish execution.
2026-04-16T18:56:14.501858418Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T18:56:14.513126597Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T18:56:14.519132385Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T18:56:14.520994662Z INFO: Collecting host service logs for crio
2026-04-16T18:56:14.521531735Z INFO: Collecting host service logs for kubelet
2026-04-16T18:56:14.521860461Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T18:56:14.522193961Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T18:56:14.522558420Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T18:56:14.522983163Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T18:56:14.523344083Z INFO: Collecting host service logs for NetworkManager
2026-04-16T18:56:14.523706516Z INFO: Collecting host service logs for openvswitch
2026-04-16T18:56:14.524013938Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T18:56:14.524341984Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T18:56:14.524715429Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T18:56:14.525110196Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T18:56:14.531431088Z INFO: Gathering HAProxy config files
2026-04-16T18:56:14.537334011Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T18:56:14.891188155Z No resources found in openshift-etcd namespace.
2026-04-16T18:56:15.010889449Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T18:56:15.395574779Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T18:56:15.395605968Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T18:56:15.539083098Z No resources found
2026-04-16T18:56:15.611919426Z Wrote inspect data to must-gather.
2026-04-16T18:56:15.630256669Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T18:56:15.634037486Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T18:56:15.679330537Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T18:56:15.736770724Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-fq9mb
2026-04-16T18:56:15.736770724Z insights-runtime-extractor-8bw6b
2026-04-16T18:56:15.736770724Z insights-runtime-extractor-9s2zk
2026-04-16T18:56:15.736770724Z insights-runtime-extractor-kwg6q
2026-04-16T18:56:15.755486014Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T18:56:15.846756186Z INFO: Worker host service log collection to complete.
2026-04-16T18:56:15.855073715Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T18:56:15.857921309Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T18:56:15.970538848Z Wrote inspect data to must-gather.
2026-04-16T18:56:16.040339896Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T18:56:16.236037210Z ERROR: No running kube-apiserver pods found
2026-04-16T18:56:16.346710629Z No resources found
2026-04-16T18:56:16.493686968Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T18:56:16.554979316Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T18:56:16.554979316Z See 'oc rsync -h' for help and examples
2026-04-16T18:56:16.600081239Z INFO: HAProxy config collection complete.
2026-04-16T18:56:16.635163540Z INFO: 'previous-logs' folder not found on ip-10-0-129-166.ec2.internal, skipping...
2026-04-16T18:56:16.690303104Z Gathering data for ns/openshift-monitoring...
2026-04-16T18:56:16.718331992Z Wrote inspect data to must-gather.
2026-04-16T18:56:16.743584351Z Wrote inspect data to must-gather.
2026-04-16T18:56:16.754228860Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T18:56:16.843196034Z Wrote inspect data to must-gather.
2026-04-16T18:56:16.867511426Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T18:56:16.885295072Z Wrote inspect data to must-gather.
2026-04-16T18:56:16.890041943Z Wrote inspect data to must-gather.
2026-04-16T18:56:16.954498082Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T18:56:17.023494518Z INFO: 'previous-logs' folder not found on ip-10-0-140-1.ec2.internal, skipping...
2026-04-16T18:56:17.060919570Z Wrote inspect data to must-gather.
2026-04-16T18:56:17.135602147Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-16T18:56:17.169603802Z Wrote inspect data to must-gather.
2026-04-16T18:56:17.289502862Z INFO: 'previous-logs' folder not found on ip-10-0-142-225.ec2.internal, skipping...
2026-04-16T18:56:17.289550869Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T18:56:17.289573301Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T18:56:17.311485988Z Wrote inspect data to must-gather.
2026-04-16T18:56:17.343326316Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T18:56:17.398604352Z Wrote inspect data to must-gather.
2026-04-16T18:56:17.541848865Z Wrote inspect data to must-gather.
2026-04-16T18:56:17.558461939Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T18:56:17.750713274Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T18:56:17.777117228Z Wrote inspect data to must-gather.
2026-04-16T18:56:17.811433631Z INFO: INTERCONNECT MODE
2026-04-16T18:56:17.811467594Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T18:56:17.960843307Z INFO: Gathering OVN_Northbound from ovnkube-node-8t42k...
2026-04-16T18:56:17.961101917Z INFO: Gathering OVN_Northbound from ovnkube-node-dsp8f...
2026-04-16T18:56:17.961312861Z INFO: Gathering OVN_Northbound from ovnkube-node-m5gbp...
2026-04-16T18:56:17.961614585Z INFO: Gathering OVN_Southbound from ovnkube-node-8t42k...
2026-04-16T18:56:17.961905877Z INFO: Gathering OVN_Southbound from ovnkube-node-dsp8f...
2026-04-16T18:56:17.962169858Z INFO: Gathering OVN_Southbound from ovnkube-node-m5gbp...
2026-04-16T18:56:17.965113488Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T18:56:17.999270887Z Wrote inspect data to must-gather.
2026-04-16T18:56:18.229258245Z tar: Removing leading `/' from member names
2026-04-16T18:56:18.240992847Z tar: Removing leading `/' from member names
2026-04-16T18:56:18.254622128Z tar: Removing leading `/' from member names
2026-04-16T18:56:18.260430656Z tar: Removing leading `/' from member names
2026-04-16T18:56:18.264946005Z tar: Removing leading `/' from member names
2026-04-16T18:56:18.282146970Z tar: Removing leading `/' from member names
2026-04-16T18:56:18.328921659Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T18:56:18.376051572Z Wrote inspect data to must-gather.
2026-04-16T18:56:18.436759837Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T18:56:18.584476170Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T18:56:18.609670687Z Wrote inspect data to must-gather.
2026-04-16T18:56:18.738667788Z INFO: Waiting for network log collection to complete ...
2026-04-16T18:56:18.738752189Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T18:56:18.739063167Z INFO: Copying ovnk databases complete.
2026-04-16T18:56:18.740565089Z 3.3M	must-gather/network_logs/ovnk_database_store
2026-04-16T18:56:18.742964921Z ovnk_database_store/
2026-04-16T18:56:18.742964921Z ovnk_database_store/ovnkube-node-8t42k_nbdb
2026-04-16T18:56:18.751154907Z ovnk_database_store/ovnkube-node-8t42k_sbdb
2026-04-16T18:56:18.768796274Z ovnk_database_store/ovnkube-node-dsp8f_nbdb
2026-04-16T18:56:18.782730158Z ovnk_database_store/ovnkube-node-m5gbp_sbdb
2026-04-16T18:56:18.800031601Z ovnk_database_store/ovnkube-node-dsp8f_sbdb
2026-04-16T18:56:18.822507518Z ovnk_database_store/ovnkube-node-m5gbp_nbdb
2026-04-16T18:56:18.832090820Z INFO: Network log collection complete.
2026-04-16T18:56:18.839543055Z Wrote inspect data to must-gather.
2026-04-16T18:56:18.962159967Z INFO: Getting status from alertmanager-main-0
2026-04-16T18:56:19.039576749Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T18:56:19.126523039Z Wrote inspect data to must-gather.
2026-04-16T18:56:19.205809507Z Gathering data for ns/openshift-network-console...
2026-04-16T18:56:19.385562686Z Wrote inspect data to must-gather.
2026-04-16T18:56:19.603944305Z Gathering data for ns/openshift-console-operator...
2026-04-16T18:56:19.654510907Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T18:56:19.658011729Z INFO: Fallback to identify the container image from release info
2026-04-16T18:56:19.946554861Z Gathering data for ns/openshift-console...
2026-04-16T18:56:20.162939921Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T18:56:20.302014184Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T18:56:20.365557516Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T18:56:20.433863913Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T18:56:20.682613435Z Gathering data for ns/openshift-dns-operator...
2026-04-16T18:56:20.965143828Z Gathering data for ns/openshift-dns...
2026-04-16T18:56:21.495395926Z Gathering data for ns/openshift-image-registry...
2026-04-16T18:56:21.563606225Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T18:56:21.928390207Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T18:56:22.143110997Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-16T18:56:22.239258068Z Gathering data for ns/openshift-ingress...
2026-04-16T18:56:22.557475384Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T18:56:22.691165192Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T18:56:22.928877055Z Gathering data for ns/openshift-insights...
2026-04-16T18:56:23.809242699Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T18:56:24.924016016Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T18:56:24.949347972Z Gathering data for ns/kserve...
2026-04-16T18:56:26.063173854Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T18:56:26.384080703Z Gathering data for ns/openshift-config...
2026-04-16T18:56:26.644601252Z Gathering data for ns/openshift-config-managed...
2026-04-16T18:56:27.065503809Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T18:56:27.147695476Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-16T18:56:27.189923235Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T18:56:27.347676874Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T18:56:27.593578908Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T18:56:27.866576087Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T18:56:28.142984894Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T18:56:28.301681409Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T18:56:28.465090171Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T18:56:28.590959952Z Collecting performance related data for node ip-10-0-142-225.ec2.internal
2026-04-16T18:56:28.597510304Z Collecting performance related data for node ip-10-0-129-166.ec2.internal
2026-04-16T18:56:28.604627044Z Collecting performance related data for node ip-10-0-140-1.ec2.internal
2026-04-16T18:56:28.829281111Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T18:56:29.190761417Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T18:56:29.601826455Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T18:56:30.279001344Z Gathering data for ns/openshift-multus...
2026-04-16T18:56:31.772360687Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T18:56:32.152159835Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-16T18:56:33.591661920Z Gathering data for ns/openshift-host-network...
2026-04-16T18:56:33.893544743Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T18:56:34.448951060Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T18:56:34.843540298Z Gathering data for ns/openshift-network-operator...
2026-04-16T18:56:35.218662490Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T18:56:35.484610287Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T18:56:35.944923342Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T18:56:36.331025870Z Collecting kubelet logs for node ip-10-0-142-225.ec2.internal
2026-04-16T18:56:36.332760496Z Collecting kubelet logs for node ip-10-0-129-166.ec2.internal
2026-04-16T18:56:36.334750290Z Collecting kubelet logs for node ip-10-0-140-1.ec2.internal
2026-04-16T18:56:36.371806433Z Gathering data for ns/openshift-apiserver...
2026-04-16T18:56:36.714917037Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T18:56:36.720543533Z INFO: Node performance data collection complete.
2026-04-16T18:56:36.820872191Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T18:56:37.086618331Z Gathering data for ns/openshift-controller-manager...
2026-04-16T18:56:37.156630311Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-16T18:56:37.388610585Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T18:56:37.997967083Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T18:56:38.265674713Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T18:56:38.582106509Z Gathering data for ns/openshift-service-ca...
2026-04-16T18:56:38.918029192Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T18:56:39.787897187Z Wrote inspect data to must-gather.
2026-04-16T18:56:39.787941421Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T18:56:39.787941421Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T18:56:39.915175806Z error: the server doesn't have a resource type "clusters"
2026-04-16T18:56:40.047079175Z Caches written to disk