2026-04-24T22:20:27.709101757Z [disk usage checker] Started
2026-04-24T22:20:27.743159724Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-24T22:20:28.044794886Z Gathering data for ns/openshift-cluster-version...
2026-04-24T22:20:28.170324677Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:20:28.261458925Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:20:28.342957851Z Gathering data for ns/default...
2026-04-24T22:20:28.628267446Z Gathering data for ns/openshift...
2026-04-24T22:20:29.022190280Z Gathering data for ns/kube-system...
2026-04-24T22:20:29.842330855Z Gathering data for ns/openshift-etcd...
2026-04-24T22:20:30.124138588Z Wrote inspect data to must-gather.
2026-04-24T22:20:30.124168218Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:20:30.124168218Z     namespaces "assisted-installer" not found
2026-04-24T22:20:30.201508557Z Waiting on subprocesses to finish execution.
2026-04-24T22:20:30.213878659Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T22:20:30.222510133Z INFO: Collecting host service logs for crio
2026-04-24T22:20:30.222855294Z INFO: Collecting host service logs for kubelet
2026-04-24T22:20:30.223082509Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T22:20:30.223152276Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T22:20:30.223438049Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T22:20:30.223723034Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T22:20:30.223828988Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T22:20:30.224029165Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T22:20:30.224312597Z INFO: Collecting host service logs for NetworkManager
2026-04-24T22:20:30.224598707Z INFO: Collecting host service logs for openvswitch
2026-04-24T22:20:30.224936335Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T22:20:30.225256763Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T22:20:30.225764763Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T22:20:30.226152469Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T22:20:30.252885295Z INFO: Gathering HAProxy config files
2026-04-24T22:20:30.382622149Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T22:20:30.724774178Z No resources found
2026-04-24T22:20:30.792340514Z No resources found in openshift-etcd namespace.
2026-04-24T22:20:30.874903579Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T22:20:31.269438783Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T22:20:31.269471158Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T22:20:31.485459091Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T22:20:31.519163229Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T22:20:31.653199299Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T22:20:31.712789305Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T22:20:31.751985843Z Gathering data for ns/openshift-monitoring...
2026-04-24T22:20:31.755768961Z No resources found
2026-04-24T22:20:31.861705913Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-xsrdx
2026-04-24T22:20:31.861803732Z insights-runtime-extractor-78x5v
2026-04-24T22:20:31.861851635Z insights-runtime-extractor-ccjvm
2026-04-24T22:20:31.861873902Z insights-runtime-extractor-n68s2
2026-04-24T22:20:31.893566161Z Wrote inspect data to must-gather.
2026-04-24T22:20:31.926572852Z Wrote inspect data to must-gather.
2026-04-24T22:20:31.974538345Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T22:20:32.048435136Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.059109344Z INFO: 'previous-logs' folder not found on ip-10-0-132-159.ec2.internal, skipping...
2026-04-24T22:20:32.087845828Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T22:20:32.260293829Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:20:32.304249486Z INFO: Worker host service log collection to complete.
2026-04-24T22:20:32.305124491Z ERROR: No running kube-apiserver pods found
2026-04-24T22:20:32.400737250Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T22:20:32.400737250Z See 'oc rsync -h' for help and examples
2026-04-24T22:20:32.412511804Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T22:20:32.483051484Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.488194633Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.501624806Z tar: Removing leading `/' from member names
2026-04-24T22:20:32.510146705Z INFO: HAProxy config collection complete.
2026-04-24T22:20:32.526957558Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T22:20:32.549814937Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.595067335Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.622780571Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.650819643Z INFO: 'previous-logs' folder not found on ip-10-0-134-147.ec2.internal, skipping...
2026-04-24T22:20:32.722424977Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T22:20:32.748454127Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-24T22:20:32.882029237Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.913198558Z INFO: 'previous-logs' folder not found on ip-10-0-134-248.ec2.internal, skipping...
2026-04-24T22:20:32.913239144Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T22:20:32.913249299Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T22:20:32.958517141Z Wrote inspect data to must-gather.
2026-04-24T22:20:32.988029010Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T22:20:33.129781085Z Wrote inspect data to must-gather.
2026-04-24T22:20:33.190935702Z Wrote inspect data to must-gather.
2026-04-24T22:20:33.370831987Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T22:20:33.412853513Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T22:20:33.441997740Z INFO: INTERCONNECT MODE
2026-04-24T22:20:33.442032398Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T22:20:33.577103806Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:20:33.607466790Z Wrote inspect data to must-gather.
2026-04-24T22:20:33.613793648Z INFO: Gathering OVN_Northbound from ovnkube-node-49kt7...
2026-04-24T22:20:33.613974250Z INFO: Gathering OVN_Northbound from ovnkube-node-dzbzn...
2026-04-24T22:20:33.614179086Z INFO: Gathering OVN_Northbound from ovnkube-node-zvr9w...
2026-04-24T22:20:33.614493042Z INFO: Gathering OVN_Southbound from ovnkube-node-49kt7...
2026-04-24T22:20:33.614893413Z INFO: Gathering OVN_Southbound from ovnkube-node-dzbzn...
2026-04-24T22:20:33.615184322Z INFO: Gathering OVN_Southbound from ovnkube-node-zvr9w...
2026-04-24T22:20:33.893580741Z tar: Removing leading `/' from member names
2026-04-24T22:20:33.925703230Z tar: Removing leading `/' from member names
2026-04-24T22:20:33.936314586Z tar: Removing leading `/' from member names
2026-04-24T22:20:33.946319861Z tar: Removing leading `/' from member names
2026-04-24T22:20:33.948985314Z tar: Removing leading `/' from member names
2026-04-24T22:20:33.963078751Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T22:20:33.972296993Z Wrote inspect data to must-gather.
2026-04-24T22:20:33.979207148Z tar: Removing leading `/' from member names
2026-04-24T22:20:34.204315065Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T22:20:34.206263709Z Wrote inspect data to must-gather.
2026-04-24T22:20:34.346194950Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:20:34.424956391Z INFO: Waiting for network log collection to complete ...
2026-04-24T22:20:34.424990959Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T22:20:34.425466854Z INFO: Copying ovnk databases complete.
2026-04-24T22:20:34.427704216Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-04-24T22:20:34.430539114Z ovnk_database_store/
2026-04-24T22:20:34.430584437Z ovnk_database_store/ovnkube-node-49kt7_sbdb
2026-04-24T22:20:34.449145173Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T22:20:34.455829946Z ovnk_database_store/ovnkube-node-zvr9w_sbdb
2026-04-24T22:20:34.478971339Z ovnk_database_store/ovnkube-node-zvr9w_nbdb
2026-04-24T22:20:34.490623472Z ovnk_database_store/ovnkube-node-dzbzn_nbdb
2026-04-24T22:20:34.500818459Z ovnk_database_store/ovnkube-node-49kt7_nbdb
2026-04-24T22:20:34.512788708Z ovnk_database_store/ovnkube-node-dzbzn_sbdb
2026-04-24T22:20:34.524424405Z Wrote inspect data to must-gather.
2026-04-24T22:20:34.538944352Z INFO: Network log collection complete.
2026-04-24T22:20:34.542989121Z Gathering data for ns/openshift-network-console...
2026-04-24T22:20:34.715101620Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T22:20:34.793295070Z Wrote inspect data to must-gather.
2026-04-24T22:20:35.032003367Z Gathering data for ns/openshift-console-operator...
2026-04-24T22:20:35.088035308Z Wrote inspect data to must-gather.
2026-04-24T22:20:35.091444723Z INFO: Getting status from alertmanager-main-0
2026-04-24T22:20:35.385456212Z Wrote inspect data to must-gather.
2026-04-24T22:20:35.429216065Z Gathering data for ns/openshift-console...
2026-04-24T22:20:35.649666958Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T22:20:35.652914242Z INFO: Fallback to identify the container image from release info
2026-04-24T22:20:35.862651697Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T22:20:36.202134960Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T22:20:36.203051923Z Gathering data for ns/openshift-dns-operator...
2026-04-24T22:20:36.333057495Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T22:20:36.476458474Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T22:20:36.510155332Z Gathering data for ns/openshift-dns...
2026-04-24T22:20:37.087422519Z Gathering data for ns/openshift-image-registry...
2026-04-24T22:20:37.551042387Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T22:20:37.626122568Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T22:20:37.755799522Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-24T22:20:37.860181564Z Gathering data for ns/openshift-ingress...
2026-04-24T22:20:38.174358466Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T22:20:38.530961839Z Gathering data for ns/openshift-insights...
2026-04-24T22:20:38.739443136Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T22:20:39.873168206Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T22:20:40.563603073Z Gathering data for ns/kserve...
2026-04-24T22:20:40.995773929Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T22:20:42.122010913Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T22:20:42.277863792Z Gathering data for ns/openshift-config...
2026-04-24T22:20:42.561338939Z Gathering data for ns/openshift-config-managed...
2026-04-24T22:20:42.761511866Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-24T22:20:42.956575731Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T22:20:43.207528030Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T22:20:43.260237441Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T22:20:43.451956574Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T22:20:43.711181738Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T22:20:43.962430582Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T22:20:44.197689552Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T22:20:44.382352741Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T22:20:44.482474003Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T22:20:44.662574679Z Collecting performance related data for node ip-10-0-134-248.ec2.internal
2026-04-24T22:20:44.670853972Z Collecting performance related data for node ip-10-0-132-159.ec2.internal
2026-04-24T22:20:44.681392696Z Collecting performance related data for node ip-10-0-134-147.ec2.internal
2026-04-24T22:20:44.847624053Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T22:20:45.174556923Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T22:20:45.810921351Z Gathering data for ns/openshift-multus...
2026-04-24T22:20:47.216548995Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T22:20:47.766167557Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-24T22:20:48.785910858Z Gathering data for ns/openshift-host-network...
2026-04-24T22:20:49.085568937Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T22:20:49.590543485Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T22:20:50.030669827Z Gathering data for ns/openshift-network-operator...
2026-04-24T22:20:50.394829154Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T22:20:50.709693446Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T22:20:51.196310183Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T22:20:51.469818445Z Gathering data for ns/openshift-apiserver...
2026-04-24T22:20:51.855002015Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T22:20:52.110351346Z Gathering data for ns/openshift-controller-manager...
2026-04-24T22:20:52.369027745Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T22:20:52.771059145Z [disk usage checker] Volume usage percentage: current = 29 ; allowed = 70
2026-04-24T22:20:52.836266768Z Collecting kubelet logs for node ip-10-0-134-248.ec2.internal
2026-04-24T22:20:52.838442361Z Collecting kubelet logs for node ip-10-0-132-159.ec2.internal
2026-04-24T22:20:52.840890656Z Collecting kubelet logs for node ip-10-0-134-147.ec2.internal
2026-04-24T22:20:53.035003178Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T22:20:53.254114149Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T22:20:53.261456276Z INFO: Node performance data collection complete.
2026-04-24T22:20:53.362338011Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T22:20:53.669310121Z Gathering data for ns/openshift-service-ca...
2026-04-24T22:20:53.985096183Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T22:20:54.803298801Z Wrote inspect data to must-gather.
2026-04-24T22:20:54.803340444Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:20:54.803340444Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T22:20:54.930025503Z error: the server doesn't have a resource type "clusters"
2026-04-24T22:20:55.080782728Z Caches written to disk