2026-05-21T21:17:43.666906335Z [disk usage checker] Started
2026-05-21T21:17:43.670519286Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-21T21:17:43.884810511Z Gathering data for ns/openshift-cluster-version...
2026-05-21T21:17:44.053769877Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T21:17:44.160666139Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T21:17:44.275262912Z Gathering data for ns/default...
2026-05-21T21:17:44.701443003Z Gathering data for ns/openshift...
2026-05-21T21:17:45.168217260Z Gathering data for ns/kube-system...
2026-05-21T21:17:45.923914435Z Gathering data for ns/openshift-etcd...
2026-05-21T21:17:46.236381983Z Waiting on subprocesses to finish execution.
2026-05-21T21:17:46.241742552Z INFO: Gathering HAProxy config files
2026-05-21T21:17:46.255457444Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-21T21:17:46.255779821Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-21T21:17:46.284521337Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-21T21:17:46.311220188Z INFO: Collecting host service logs for crio
2026-05-21T21:17:46.311558191Z INFO: Collecting host service logs for kubelet
2026-05-21T21:17:46.311871594Z INFO: Collecting host service logs for rpm-ostreed
2026-05-21T21:17:46.312172737Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-21T21:17:46.312495508Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-21T21:17:46.312791740Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-21T21:17:46.313078022Z INFO: Collecting host service logs for NetworkManager
2026-05-21T21:17:46.313393375Z INFO: Collecting host service logs for openvswitch
2026-05-21T21:17:46.313699053Z INFO: Collecting host service logs for ovs-configuration
2026-05-21T21:17:46.313983682Z INFO: Collecting host service logs for ovsdb-server
2026-05-21T21:17:46.314252032Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-21T21:17:46.314630945Z INFO: Waiting for worker host service log collection to complete ...
2026-05-21T21:17:46.344589719Z Wrote inspect data to must-gather.
2026-05-21T21:17:46.344687288Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T21:17:46.344687288Z     namespaces "assisted-installer" not found
2026-05-21T21:17:46.347124601Z INFO: Waiting for node performance related collection to complete ...
2026-05-21T21:17:46.808323463Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-r9kfc
2026-05-21T21:17:46.808375168Z insights-runtime-extractor-8d4sk
2026-05-21T21:17:46.808411505Z insights-runtime-extractor-cpl7g
2026-05-21T21:17:46.808435906Z insights-runtime-extractor-xqzpt
2026-05-21T21:17:47.153712599Z No resources found in openshift-etcd namespace.
2026-05-21T21:17:47.181839123Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-21T21:17:47.213986212Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-21T21:17:47.251211024Z INFO: "metallb-operator" not detected. Skipping.
2026-05-21T21:17:47.278216286Z error: the server doesn't have a resource type "performanceprofile"
2026-05-21T21:17:47.297759863Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-21T21:17:47.527826848Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-21T21:17:47.585687674Z Wrote inspect data to must-gather.
2026-05-21T21:17:47.595581714Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-21T21:17:47.595581714Z INFO: on-disk MachineConfig config collection complete.
2026-05-21T21:17:47.776388461Z No resources found
2026-05-21T21:17:47.836298367Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-21T21:17:47.841195651Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-21T21:17:47.841195651Z See 'oc rsync -h' for help and examples
2026-05-21T21:17:47.882468949Z INFO: Worker host service log collection to complete.
2026-05-21T21:17:47.987154865Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-21T21:17:48.140020791Z ERROR: No running kube-apiserver pods found
2026-05-21T21:17:48.214308041Z tar: Removing leading `/' from member names
2026-05-21T21:17:48.221320280Z INFO: HAProxy config collection complete.
2026-05-21T21:17:48.495138176Z Wrote inspect data to must-gather.
2026-05-21T21:17:48.726231969Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-21T21:17:48.867559361Z No resources found
2026-05-21T21:17:48.880839768Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-21T21:17:48.887731435Z Gathering data for ns/kuadrant-system...
2026-05-21T21:17:48.898756162Z Wrote inspect data to must-gather.
2026-05-21T21:17:49.035528333Z Wrote inspect data to must-gather.
2026-05-21T21:17:49.063974925Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T21:17:49.180216349Z INFO: 'previous-logs' folder not found on ip-10-0-131-158.ec2.internal, skipping...
2026-05-21T21:17:49.205109865Z error: the server doesn't have a resource type "machineconfigs"
2026-05-21T21:17:49.253895460Z Wrote inspect data to must-gather.
2026-05-21T21:17:49.288663805Z Wrote inspect data to must-gather.
2026-05-21T21:17:49.290604900Z INFO: Getting rules from prometheus-k8s-0
2026-05-21T21:17:49.540380430Z Wrote inspect data to must-gather.
2026-05-21T21:17:49.597741630Z INFO: 'previous-logs' folder not found on ip-10-0-132-7.ec2.internal, skipping...
2026-05-21T21:17:49.692554960Z Wrote inspect data to must-gather.
2026-05-21T21:17:49.807090171Z INFO: Getting status/config from prometheus-k8s-0
2026-05-21T21:17:49.889485190Z Wrote inspect data to must-gather.
2026-05-21T21:17:50.080528787Z INFO: 'previous-logs' folder not found on ip-10-0-133-110.ec2.internal, skipping...
2026-05-21T21:17:50.080570544Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-21T21:17:50.080606935Z INFO: Machine Config Daemon termination log collection complete.
2026-05-21T21:17:50.152732831Z Wrote inspect data to must-gather.
2026-05-21T21:17:50.179660653Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-21T21:17:50.249072543Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-21T21:17:50.322011527Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-21T21:17:50.506535215Z Wrote inspect data to must-gather.
2026-05-21T21:17:50.521039971Z Wrote inspect data to must-gather.
2026-05-21T21:17:50.612860115Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-21T21:17:50.798879283Z W0521 21:17:50.798819    1157 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-05-21T21:17:50.825895401Z Wrote inspect data to must-gather.
2026-05-21T21:17:50.849134647Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-21T21:17:50.894876623Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-21T21:17:50.923282786Z INFO: INTERCONNECT MODE
2026-05-21T21:17:50.923377883Z INFO: Gathering ovn-kubernetes DBs
2026-05-21T21:17:51.166151696Z Gathering data for ns/openshift-monitoring...
2026-05-21T21:17:51.192838640Z INFO: Gathering OVN_Northbound from ovnkube-node-5zmnk...
2026-05-21T21:17:51.193133690Z INFO: Gathering OVN_Northbound from ovnkube-node-7krlb...
2026-05-21T21:17:51.193409776Z INFO: Gathering OVN_Northbound from ovnkube-node-v7n7h...
2026-05-21T21:17:51.193719888Z INFO: Gathering OVN_Southbound from ovnkube-node-5zmnk...
2026-05-21T21:17:51.193976568Z INFO: Gathering OVN_Southbound from ovnkube-node-7krlb...
2026-05-21T21:17:51.194380083Z INFO: Gathering OVN_Southbound from ovnkube-node-v7n7h...
2026-05-21T21:17:51.271566132Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-21T21:17:51.351776062Z Wrote inspect data to must-gather.
2026-05-21T21:17:51.514301863Z tar: Removing leading `/' from member names
2026-05-21T21:17:51.524126722Z tar: Removing leading `/' from member names
2026-05-21T21:17:51.591483857Z tar: Removing leading `/' from member names
2026-05-21T21:17:51.591483857Z tar: Removing leading `/' from member names
2026-05-21T21:17:51.661371886Z tar: Removing leading `/' from member names
2026-05-21T21:17:51.678790543Z tar: Removing leading `/' from member names
2026-05-21T21:17:51.833445630Z Wrote inspect data to must-gather.
2026-05-21T21:17:51.848888067Z INFO: Getting status from alertmanager-main-0
2026-05-21T21:17:52.199059655Z Wrote inspect data to must-gather.
2026-05-21T21:17:52.271993605Z INFO: Waiting for network log collection to complete ...
2026-05-21T21:17:52.272042323Z INFO: Waiting for ovnk database copies to complete ...
2026-05-21T21:17:52.272619493Z INFO: Copying ovnk databases complete.
2026-05-21T21:17:52.274514314Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-05-21T21:17:52.277183793Z ovnk_database_store/
2026-05-21T21:17:52.277183793Z ovnk_database_store/ovnkube-node-v7n7h_nbdb
2026-05-21T21:17:52.289352849Z ovnk_database_store/ovnkube-node-5zmnk_sbdb
2026-05-21T21:17:52.321731493Z ovnk_database_store/ovnkube-node-7krlb_sbdb
2026-05-21T21:17:52.347160799Z ovnk_database_store/ovnkube-node-7krlb_nbdb
2026-05-21T21:17:52.358196279Z ovnk_database_store/ovnkube-node-5zmnk_nbdb
2026-05-21T21:17:52.376574916Z ovnk_database_store/ovnkube-node-v7n7h_sbdb
2026-05-21T21:17:52.410296888Z INFO: Network log collection complete.
2026-05-21T21:17:52.574319638Z Wrote inspect data to must-gather.
2026-05-21T21:17:52.878033363Z Wrote inspect data to must-gather.
2026-05-21T21:17:53.178888999Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-21T21:17:53.182145795Z INFO: Fallback to identify the container image from release info
2026-05-21T21:17:53.440474108Z Gathering data for ns/openshift-network-console...
2026-05-21T21:17:53.736316573Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-21T21:17:53.870258067Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-21T21:17:53.971484614Z Gathering data for ns/openshift-console-operator...
2026-05-21T21:17:54.020820537Z daemonset.apps/perf-node-gather-daemonset created
2026-05-21T21:17:54.192969068Z Waiting for performance profile collector pods to become ready: 1
2026-05-21T21:17:54.451029881Z Gathering data for ns/openshift-console...
2026-05-21T21:17:55.002522744Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-21T21:17:55.331794907Z Waiting for performance profile collector pods to become ready: 2
2026-05-21T21:17:55.493621266Z Gathering data for ns/openshift-dns-operator...
2026-05-21T21:17:55.859011082Z Gathering data for ns/openshift-dns...
2026-05-21T21:17:56.001190969Z Wrote inspect data to must-gather.
2026-05-21T21:17:56.467672619Z Waiting for performance profile collector pods to become ready: 3
2026-05-21T21:17:56.590923229Z Gathering data for ns/openshift-image-registry...
2026-05-21T21:17:57.132935326Z Gathering data for ns/openshift-ingress-operator...
2026-05-21T21:17:57.509992022Z Gathering data for ns/openshift-ingress...
2026-05-21T21:17:57.583777434Z Waiting for performance profile collector pods to become ready: 4
2026-05-21T21:17:58.353590926Z Gathering data for ns/openshift-ingress-canary...
2026-05-21T21:17:58.726051890Z Waiting for performance profile collector pods to become ready: 5
2026-05-21T21:17:58.742431407Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-21T21:17:58.818302453Z Gathering data for ns/openshift-insights...
2026-05-21T21:17:59.842102395Z Waiting for performance profile collector pods to become ready: 6
2026-05-21T21:18:00.785584123Z Gathering data for ns/opendatahub...
2026-05-21T21:18:00.960481119Z Waiting for performance profile collector pods to become ready: 7
2026-05-21T21:18:02.078514802Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-21T21:18:02.299136455Z Gathering data for ns/openshift-lws-operator...
2026-05-21T21:18:02.321259141Z Collecting performance related data for node ip-10-0-131-158.ec2.internal
2026-05-21T21:18:02.327885055Z Collecting performance related data for node ip-10-0-133-110.ec2.internal
2026-05-21T21:18:02.335192317Z Collecting performance related data for node ip-10-0-132-7.ec2.internal
2026-05-21T21:18:03.750547242Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-21T21:18:03.804351944Z Gathering data for ns/openshift-config...
2026-05-21T21:18:04.127579251Z Gathering data for ns/openshift-config-managed...
2026-05-21T21:18:04.614756346Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-21T21:18:04.964875235Z Gathering data for ns/openshift-kube-apiserver...
2026-05-21T21:18:05.308520196Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-21T21:18:05.663840720Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-21T21:18:06.017839546Z Gathering data for ns/openshift-kube-scheduler...
2026-05-21T21:18:06.338597554Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-21T21:18:06.669016722Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-21T21:18:07.050881546Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-21T21:18:07.406488134Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-21T21:18:08.089983730Z Gathering data for ns/openshift-multus...
2026-05-21T21:18:08.755612346Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-21T21:18:09.750266330Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-21T21:18:11.149390239Z Collecting kubelet logs for node ip-10-0-131-158.ec2.internal
2026-05-21T21:18:11.151750017Z Collecting kubelet logs for node ip-10-0-133-110.ec2.internal
2026-05-21T21:18:11.153566135Z Collecting kubelet logs for node ip-10-0-132-7.ec2.internal
2026-05-21T21:18:11.451029731Z Gathering data for ns/openshift-host-network...
2026-05-21T21:18:11.558675535Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-21T21:18:11.564564842Z INFO: Node performance data collection complete.
2026-05-21T21:18:11.819034661Z Gathering data for ns/openshift-network-diagnostics...
2026-05-21T21:18:12.365303333Z Gathering data for ns/openshift-network-node-identity...
2026-05-21T21:18:12.846078103Z Gathering data for ns/openshift-network-operator...
2026-05-21T21:18:13.222117850Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-21T21:18:13.522088621Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-21T21:18:13.760514333Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-05-21T21:18:13.977135863Z Gathering data for ns/openshift-apiserver-operator...
2026-05-21T21:18:14.311676877Z Gathering data for ns/openshift-apiserver...
2026-05-21T21:18:14.757633609Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-21T21:18:15.137338501Z Gathering data for ns/openshift-controller-manager...
2026-05-21T21:18:15.433031745Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-21T21:18:16.091634000Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-21T21:18:16.400182879Z Gathering data for ns/openshift-service-ca-operator...
2026-05-21T21:18:16.724728514Z Gathering data for ns/openshift-service-ca...
2026-05-21T21:18:17.114626547Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-21T21:18:18.073638428Z Wrote inspect data to must-gather.
2026-05-21T21:18:18.073684927Z error: inspection completed with the errors occurred while gathering data:
2026-05-21T21:18:18.073684927Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-21T21:18:18.208623336Z error: the server doesn't have a resource type "clusters"
2026-05-21T21:18:18.373451636Z Caches written to disk