2026-05-21T20:57:19.900105755Z [disk usage checker] Started
2026-05-21T20:57:19.903848318Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-05-21T20:57:20.161974260Z Gathering data for ns/openshift-cluster-version...
2026-05-21T20:57:20.315976102Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T20:57:20.410075667Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T20:57:20.511752585Z Gathering data for ns/default...
2026-05-21T20:57:20.872164118Z Gathering data for ns/openshift...
2026-05-21T20:57:21.310827791Z Gathering data for ns/kube-system...
2026-05-21T20:57:22.031797696Z Gathering data for ns/openshift-etcd...
2026-05-21T20:57:22.378500791Z Wrote inspect data to must-gather.
2026-05-21T20:57:22.378539240Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T20:57:22.378539240Z     namespaces "assisted-installer" not found
2026-05-21T20:57:22.441198810Z Waiting on subprocesses to finish execution.
2026-05-21T20:57:22.451826026Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-21T20:57:22.452051329Z INFO: Collecting host service logs for crio
2026-05-21T20:57:22.452379525Z INFO: Collecting host service logs for kubelet
2026-05-21T20:57:22.452699476Z INFO: Collecting host service logs for rpm-ostreed
2026-05-21T20:57:22.452996627Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-21T20:57:22.452996627Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-21T20:57:22.453250119Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-21T20:57:22.453527986Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-21T20:57:22.454689888Z INFO: Collecting host service logs for NetworkManager
2026-05-21T20:57:22.455006722Z INFO: Collecting host service logs for openvswitch
2026-05-21T20:57:22.455290711Z INFO: Collecting host service logs for ovs-configuration
2026-05-21T20:57:22.455603569Z INFO: Collecting host service logs for ovsdb-server
2026-05-21T20:57:22.455903203Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-21T20:57:22.456230906Z INFO: Waiting for worker host service log collection to complete ...
2026-05-21T20:57:22.467149308Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-21T20:57:22.470486315Z INFO: Gathering HAProxy config files
2026-05-21T20:57:22.570794373Z INFO: Waiting for node performance related collection to complete ...
2026-05-21T20:57:23.102227036Z No resources found
2026-05-21T20:57:23.228857309Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-zzz86
2026-05-21T20:57:23.228924340Z insights-runtime-extractor-cnhtn
2026-05-21T20:57:23.228947221Z insights-runtime-extractor-g96xj
2026-05-21T20:57:23.228966978Z insights-runtime-extractor-vb6w7
2026-05-21T20:57:23.280424251Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-21T20:57:23.280524018Z INFO: on-disk MachineConfig config collection complete.
2026-05-21T20:57:23.357312103Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-21T20:57:23.432717870Z Wrote inspect data to must-gather.
2026-05-21T20:57:23.527768744Z No resources found in openshift-etcd namespace.
2026-05-21T20:57:23.644986853Z INFO: "metallb-operator" not detected. Skipping.
2026-05-21T20:57:23.795154000Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-21T20:57:23.795154000Z See 'oc rsync -h' for help and examples
2026-05-21T20:57:23.811492590Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-21T20:57:23.930266069Z error: the server doesn't have a resource type "performanceprofile"
2026-05-21T20:57:23.998630024Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-21T20:57:24.131756338Z INFO: Worker host service log collection to complete.
2026-05-21T20:57:24.135640376Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-21T20:57:24.262893524Z ERROR: No running kube-apiserver pods found
2026-05-21T20:57:24.399866523Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-21T20:57:24.641430957Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-21T20:57:24.659460892Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-21T20:57:24.732054228Z No resources found
2026-05-21T20:57:24.851388292Z Wrote inspect data to must-gather.
2026-05-21T20:57:24.871992146Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T20:57:24.924915489Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-05-21T20:57:25.051646770Z Wrote inspect data to must-gather.
2026-05-21T20:57:25.095001343Z Gathering data for ns/kuadrant-system...
2026-05-21T20:57:25.107265621Z tar: Removing leading `/' from member names
2026-05-21T20:57:25.114042712Z INFO: HAProxy config collection complete.
2026-05-21T20:57:25.189464637Z Wrote inspect data to must-gather.
2026-05-21T20:57:25.203702101Z INFO: 'previous-logs' folder not found on ip-10-0-130-100.ec2.internal, skipping...
2026-05-21T20:57:25.269506616Z Wrote inspect data to must-gather.
2026-05-21T20:57:25.336578124Z Wrote inspect data to must-gather.
2026-05-21T20:57:25.476485173Z error: the server doesn't have a resource type "machineconfigs"
2026-05-21T20:57:25.584330133Z INFO: 'previous-logs' folder not found on ip-10-0-130-74.ec2.internal, skipping...
2026-05-21T20:57:25.603831086Z INFO: Getting rules from prometheus-k8s-0
2026-05-21T20:57:25.751462055Z Wrote inspect data to must-gather.
2026-05-21T20:57:25.925110785Z Wrote inspect data to must-gather.
2026-05-21T20:57:25.973871987Z INFO: 'previous-logs' folder not found on ip-10-0-134-40.ec2.internal, skipping...
2026-05-21T20:57:25.973974994Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-21T20:57:25.974026265Z INFO: Machine Config Daemon termination log collection complete.
2026-05-21T20:57:26.106803853Z Wrote inspect data to must-gather.
2026-05-21T20:57:26.187820609Z INFO: Getting status/config from prometheus-k8s-0
2026-05-21T20:57:26.257668567Z Wrote inspect data to must-gather.
2026-05-21T20:57:26.410587884Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T20:57:26.490894791Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-21T20:57:26.498079721Z Wrote inspect data to must-gather.
2026-05-21T20:57:26.731501569Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-21T20:57:26.812195002Z Wrote inspect data to must-gather.
2026-05-21T20:57:26.930677273Z INFO: INTERCONNECT MODE
2026-05-21T20:57:26.930783459Z INFO: Gathering ovn-kubernetes DBs
2026-05-21T20:57:26.978413315Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T20:57:27.031097289Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-21T20:57:27.200026497Z INFO: Gathering OVN_Northbound from ovnkube-node-49t4r...
2026-05-21T20:57:27.200337908Z INFO: Gathering OVN_Northbound from ovnkube-node-6tkgs...
2026-05-21T20:57:27.200627190Z INFO: Gathering OVN_Northbound from ovnkube-node-tcncm...
2026-05-21T20:57:27.200926294Z INFO: Gathering OVN_Southbound from ovnkube-node-49t4r...
2026-05-21T20:57:27.201356059Z INFO: Gathering OVN_Southbound from ovnkube-node-6tkgs...
2026-05-21T20:57:27.201631522Z INFO: Gathering OVN_Southbound from ovnkube-node-tcncm...
2026-05-21T20:57:27.217722727Z Wrote inspect data to must-gather.
2026-05-21T20:57:27.223122547Z Gathering data for ns/openshift-monitoring...
2026-05-21T20:57:27.399324277Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-21T20:57:27.620726361Z tar: Removing leading `/' from member names
2026-05-21T20:57:27.631245663Z tar: Removing leading `/' from member names
2026-05-21T20:57:27.693882154Z tar: Removing leading `/' from member names
2026-05-21T20:57:27.706176477Z tar: Removing leading `/' from member names
2026-05-21T20:57:27.710634356Z tar: Removing leading `/' from member names
2026-05-21T20:57:27.724873515Z tar: Removing leading `/' from member names
2026-05-21T20:57:27.843393859Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-21T20:57:27.848204595Z Wrote inspect data to must-gather.
2026-05-21T20:57:28.277183381Z Wrote inspect data to must-gather.
2026-05-21T20:57:28.322873328Z INFO: Getting status from alertmanager-main-0
2026-05-21T20:57:28.352876029Z INFO: Waiting for network log collection to complete ...
2026-05-21T20:57:28.353796442Z INFO: Waiting for ovnk database copies to complete ...
2026-05-21T20:57:28.354303361Z INFO: Copying ovnk databases complete.
2026-05-21T20:57:28.356918103Z 4.3M	must-gather/network_logs/ovnk_database_store
2026-05-21T20:57:28.360065110Z ovnk_database_store/
2026-05-21T20:57:28.360065110Z ovnk_database_store/ovnkube-node-49t4r_sbdb
2026-05-21T20:57:28.388186911Z ovnk_database_store/ovnkube-node-49t4r_nbdb
2026-05-21T20:57:28.398654128Z ovnk_database_store/ovnkube-node-tcncm_sbdb
2026-05-21T20:57:28.426067267Z ovnk_database_store/ovnkube-node-tcncm_nbdb
2026-05-21T20:57:28.440980382Z ovnk_database_store/ovnkube-node-6tkgs_nbdb
2026-05-21T20:57:28.451991088Z ovnk_database_store/ovnkube-node-6tkgs_sbdb
2026-05-21T20:57:28.484903628Z INFO: Network log collection complete.
2026-05-21T20:57:28.633983396Z Wrote inspect data to must-gather.
2026-05-21T20:57:28.974137896Z Wrote inspect data to must-gather.
2026-05-21T20:57:29.285431756Z Wrote inspect data to must-gather.
2026-05-21T20:57:29.585712126Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-21T20:57:29.588031243Z INFO: Fallback to identify the container image from release info
2026-05-21T20:57:29.655083736Z Gathering data for ns/openshift-network-console...
2026-05-21T20:57:29.948687582Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-05-21T20:57:30.168606953Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-21T20:57:30.187460055Z Gathering data for ns/openshift-console-operator...
2026-05-21T20:57:30.305336262Z daemonset.apps/perf-node-gather-daemonset created
2026-05-21T20:57:30.443939400Z Waiting for performance profile collector pods to become ready: 1
2026-05-21T20:57:30.683479491Z Gathering data for ns/openshift-console...
2026-05-21T20:57:31.219472973Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-21T20:57:31.606071547Z Waiting for performance profile collector pods to become ready: 2
2026-05-21T20:57:31.665605308Z Gathering data for ns/openshift-dns-operator...
2026-05-21T20:57:32.020815252Z Gathering data for ns/openshift-dns...
2026-05-21T20:57:32.085429303Z Wrote inspect data to must-gather.
2026-05-21T20:57:32.605240208Z Gathering data for ns/openshift-image-registry...
2026-05-21T20:57:32.721876810Z Waiting for performance profile collector pods to become ready: 3
2026-05-21T20:57:33.083459575Z Gathering data for ns/openshift-ingress-operator...
2026-05-21T20:57:33.489099284Z Gathering data for ns/openshift-ingress...
2026-05-21T20:57:33.854721625Z Waiting for performance profile collector pods to become ready: 4
2026-05-21T20:57:34.286434002Z Gathering data for ns/openshift-ingress-canary...
2026-05-21T20:57:34.719335089Z Gathering data for ns/openshift-insights...
2026-05-21T20:57:34.958826228Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-05-21T20:57:34.977404421Z Waiting for performance profile collector pods to become ready: 5
2026-05-21T20:57:36.090544776Z Waiting for performance profile collector pods to become ready: 6
2026-05-21T20:57:36.689879461Z Gathering data for ns/opendatahub...
2026-05-21T20:57:37.196500975Z Waiting for performance profile collector pods to become ready: 7
2026-05-21T20:57:38.168632457Z Gathering data for ns/openshift-lws-operator...
2026-05-21T20:57:38.334403765Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-21T20:57:38.597711647Z Collecting performance related data for node ip-10-0-134-40.ec2.internal
2026-05-21T20:57:38.606024056Z Collecting performance related data for node ip-10-0-130-100.ec2.internal
2026-05-21T20:57:38.620000306Z Collecting performance related data for node ip-10-0-130-74.ec2.internal
2026-05-21T20:57:39.607660494Z Gathering data for ns/openshift-config...
2026-05-21T20:57:39.967792382Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-05-21T20:57:40.012652450Z Gathering data for ns/openshift-config-managed...
2026-05-21T20:57:40.486363589Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-21T20:57:40.799086010Z Gathering data for ns/openshift-kube-apiserver...
2026-05-21T20:57:41.126811918Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-21T20:57:41.500350223Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-21T20:57:41.856754117Z Gathering data for ns/openshift-kube-scheduler...
2026-05-21T20:57:42.156217694Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-21T20:57:42.499952026Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-21T20:57:42.849336074Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-21T20:57:43.189525849Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-21T20:57:43.836920890Z Gathering data for ns/openshift-multus...
2026-05-21T20:57:44.973318393Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-05-21T20:57:45.248392438Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-21T20:57:46.794086876Z Gathering data for ns/openshift-host-network...
2026-05-21T20:57:47.144418856Z Gathering data for ns/openshift-network-diagnostics...
2026-05-21T20:57:47.470967852Z Collecting kubelet logs for node ip-10-0-134-40.ec2.internal
2026-05-21T20:57:47.472791429Z Collecting kubelet logs for node ip-10-0-130-100.ec2.internal
2026-05-21T20:57:47.474642924Z Collecting kubelet logs for node ip-10-0-130-74.ec2.internal
2026-05-21T20:57:47.871683224Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-21T20:57:47.876874578Z INFO: Node performance data collection complete.
2026-05-21T20:57:47.907949950Z Gathering data for ns/openshift-network-node-identity...
2026-05-21T20:57:48.387067426Z Gathering data for ns/openshift-network-operator...
2026-05-21T20:57:48.770653743Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-21T20:57:49.080318598Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-21T20:57:49.535180792Z Gathering data for ns/openshift-apiserver-operator...
2026-05-21T20:57:49.904342931Z Gathering data for ns/openshift-apiserver...
2026-05-21T20:57:49.979932934Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-05-21T20:57:50.343680838Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-21T20:57:50.648593513Z Gathering data for ns/openshift-controller-manager...
2026-05-21T20:57:50.973240421Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-21T20:57:51.623073220Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-21T20:57:51.924988020Z Gathering data for ns/openshift-service-ca-operator...
2026-05-21T20:57:52.248722734Z Gathering data for ns/openshift-service-ca...
2026-05-21T20:57:52.624216032Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-21T20:57:53.587434799Z Wrote inspect data to must-gather.
2026-05-21T20:57:53.587481292Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T20:57:53.587481292Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-21T20:57:53.720596391Z error: the server doesn't have a resource type "clusters"
2026-05-21T20:57:53.909310636Z Caches written to disk