2026-04-16T16:36:42.976378239Z [disk usage checker] Started
2026-04-16T16:36:43.010090217Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-04-16T16:36:43.190297948Z Gathering data for ns/openshift-cluster-version...
2026-04-16T16:36:43.337685809Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T16:36:43.437763947Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T16:36:43.526658024Z Gathering data for ns/default...
2026-04-16T16:36:43.878423366Z Gathering data for ns/openshift...
2026-04-16T16:36:44.337336360Z Gathering data for ns/kube-system...
2026-04-16T16:36:44.994658467Z Gathering data for ns/openshift-etcd...
2026-04-16T16:36:45.362001724Z Wrote inspect data to must-gather.
2026-04-16T16:36:45.362036350Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T16:36:45.362036350Z     namespaces "assisted-installer" not found
2026-04-16T16:36:45.487349231Z Waiting on subprocesses to finish execution.
2026-04-16T16:36:45.496473085Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T16:36:45.497196651Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T16:36:45.499149207Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T16:36:45.503642598Z INFO: Collecting host service logs for crio
2026-04-16T16:36:45.503960806Z INFO: Collecting host service logs for kubelet
2026-04-16T16:36:45.504311620Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T16:36:45.504703021Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T16:36:45.505175502Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T16:36:45.505520014Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T16:36:45.505853156Z INFO: Collecting host service logs for NetworkManager
2026-04-16T16:36:45.506171654Z INFO: Collecting host service logs for openvswitch
2026-04-16T16:36:45.506514247Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T16:36:45.507014058Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T16:36:45.507342344Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T16:36:45.507763482Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T16:36:45.527911267Z INFO: Gathering HAProxy config files
2026-04-16T16:36:45.859181316Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T16:36:46.188907993Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T16:36:46.315173886Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T16:36:46.315256110Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T16:36:46.443158980Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T16:36:46.516552029Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T16:36:46.528977088Z Wrote inspect data to must-gather.
2026-04-16T16:36:46.550063137Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T16:36:46.653786429Z No resources found in openshift-etcd namespace.
2026-04-16T16:36:46.702262440Z No resources found
2026-04-16T16:36:46.757805391Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T16:36:46.798925099Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-t2wfh
2026-04-16T16:36:46.798985350Z insights-runtime-extractor-gjzps
2026-04-16T16:36:46.799006270Z insights-runtime-extractor-l8wbj
2026-04-16T16:36:46.799025515Z insights-runtime-extractor-x7lk6
2026-04-16T16:36:46.843125022Z INFO: Worker host service log collection to complete.
2026-04-16T16:36:46.939693967Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T16:36:47.027851952Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T16:36:47.108338589Z ERROR: No running kube-apiserver pods found
2026-04-16T16:36:47.493215652Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T16:36:47.493215652Z See 'oc rsync -h' for help and examples
2026-04-16T16:36:47.547055887Z Wrote inspect data to must-gather.
2026-04-16T16:36:47.820664878Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T16:36:47.821323461Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T16:36:47.861877968Z Wrote inspect data to must-gather.
2026-04-16T16:36:48.033076512Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-04-16T16:36:48.058350455Z INFO: 'previous-logs' folder not found on ip-10-0-131-24.ec2.internal, skipping...
2026-04-16T16:36:48.075246435Z No resources found
2026-04-16T16:36:48.146851206Z Wrote inspect data to must-gather.
2026-04-16T16:36:48.161256086Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T16:36:48.247881793Z Wrote inspect data to must-gather.
2026-04-16T16:36:48.294164560Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T16:36:48.335983535Z Wrote inspect data to must-gather.
2026-04-16T16:36:48.380156920Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T16:36:48.381834821Z Gathering data for ns/kuadrant-system...
2026-04-16T16:36:48.462109095Z tar: Removing leading `/' from member names
2026-04-16T16:36:48.468727334Z INFO: HAProxy config collection complete.
2026-04-16T16:36:48.605992773Z INFO: 'previous-logs' folder not found on ip-10-0-134-41.ec2.internal, skipping...
2026-04-16T16:36:48.714996781Z Wrote inspect data to must-gather.
2026-04-16T16:36:48.773227001Z Wrote inspect data to must-gather.
2026-04-16T16:36:48.892030899Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T16:36:48.971090866Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T16:36:48.991185514Z INFO: 'previous-logs' folder not found on ip-10-0-140-78.ec2.internal, skipping...
2026-04-16T16:36:48.991303575Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T16:36:48.991324653Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T16:36:49.114927781Z Wrote inspect data to must-gather.
2026-04-16T16:36:49.189100502Z Wrote inspect data to must-gather.
2026-04-16T16:36:49.278917378Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T16:36:49.390896796Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T16:36:49.427733005Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T16:36:49.526781582Z Gathering data for ns/openshift-monitoring...
2026-04-16T16:36:49.591073065Z Wrote inspect data to must-gather.
2026-04-16T16:36:49.725998048Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T16:36:49.752559240Z Wrote inspect data to must-gather.
2026-04-16T16:36:49.935424340Z INFO: INTERCONNECT MODE
2026-04-16T16:36:49.935521493Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T16:36:49.994972641Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T16:36:50.071329209Z Wrote inspect data to must-gather.
2026-04-16T16:36:50.128472167Z INFO: Gathering OVN_Northbound from ovnkube-node-65x5s...
2026-04-16T16:36:50.128796825Z INFO: Gathering OVN_Northbound from ovnkube-node-9n7wn...
2026-04-16T16:36:50.129067514Z INFO: Gathering OVN_Northbound from ovnkube-node-grb4s...
2026-04-16T16:36:50.129355747Z INFO: Gathering OVN_Southbound from ovnkube-node-65x5s...
2026-04-16T16:36:50.129635647Z INFO: Gathering OVN_Southbound from ovnkube-node-9n7wn...
2026-04-16T16:36:50.129892903Z INFO: Gathering OVN_Southbound from ovnkube-node-grb4s...
2026-04-16T16:36:50.348114332Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T16:36:50.461111745Z tar: Removing leading `/' from member names
2026-04-16T16:36:50.516550835Z tar: Removing leading `/' from member names
2026-04-16T16:36:50.533197736Z Wrote inspect data to must-gather.
2026-04-16T16:36:50.540720099Z tar: Removing leading `/' from member names
2026-04-16T16:36:50.541546559Z tar: Removing leading `/' from member names
2026-04-16T16:36:50.577958375Z tar: Removing leading `/' from member names
2026-04-16T16:36:50.591031951Z tar: Removing leading `/' from member names
2026-04-16T16:36:50.852670160Z Wrote inspect data to must-gather.
2026-04-16T16:36:50.933510526Z INFO: Getting status from alertmanager-main-0
2026-04-16T16:36:51.176868638Z Wrote inspect data to must-gather.
2026-04-16T16:36:51.215609615Z INFO: Waiting for network log collection to complete ...
2026-04-16T16:36:51.215664790Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T16:36:51.216190704Z INFO: Copying ovnk databases complete.
2026-04-16T16:36:51.217809976Z 5.0M	must-gather/network_logs/ovnk_database_store
2026-04-16T16:36:51.220182565Z ovnk_database_store/
2026-04-16T16:36:51.220182565Z ovnk_database_store/ovnkube-node-grb4s_nbdb
2026-04-16T16:36:51.232824653Z ovnk_database_store/ovnkube-node-65x5s_nbdb
2026-04-16T16:36:51.242643103Z ovnk_database_store/ovnkube-node-grb4s_sbdb
2026-04-16T16:36:51.273338575Z ovnk_database_store/ovnkube-node-9n7wn_sbdb
2026-04-16T16:36:51.303825767Z ovnk_database_store/ovnkube-node-9n7wn_nbdb
2026-04-16T16:36:51.318117406Z ovnk_database_store/ovnkube-node-65x5s_sbdb
2026-04-16T16:36:51.352760586Z INFO: Network log collection complete.
2026-04-16T16:36:51.454517240Z Wrote inspect data to must-gather.
2026-04-16T16:36:51.797299991Z Wrote inspect data to must-gather.
2026-04-16T16:36:51.890662279Z Gathering data for ns/openshift-network-console...
2026-04-16T16:36:52.102127079Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T16:36:52.104841251Z INFO: Fallback to identify the container image from release info
2026-04-16T16:36:52.389612162Z Gathering data for ns/openshift-console-operator...
2026-04-16T16:36:52.727511395Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T16:36:52.838698825Z Gathering data for ns/openshift-console...
2026-04-16T16:36:52.856890565Z Wrote inspect data to must-gather.
2026-04-16T16:36:52.864130311Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T16:36:53.005985182Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T16:36:53.044278103Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-04-16T16:36:53.344637627Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T16:36:53.711028286Z Gathering data for ns/openshift-dns-operator...
2026-04-16T16:36:54.028035939Z Gathering data for ns/openshift-dns...
2026-04-16T16:36:54.124064778Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T16:36:54.639986961Z Gathering data for ns/openshift-image-registry...
2026-04-16T16:36:55.099488322Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T16:36:55.249784723Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T16:36:55.480873678Z Gathering data for ns/openshift-ingress...
2026-04-16T16:36:56.032923325Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T16:36:56.379842648Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T16:36:56.499069734Z Gathering data for ns/openshift-insights...
2026-04-16T16:36:57.498052748Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T16:36:58.049460031Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-04-16T16:36:58.610137799Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T16:36:59.148706518Z Gathering data for ns/openshift-lws-operator...
2026-04-16T16:36:59.702360427Z Gathering data for ns/kserve...
2026-04-16T16:36:59.735920319Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T16:37:00.855033590Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T16:37:01.115212455Z Collecting performance related data for node ip-10-0-134-41.ec2.internal
2026-04-16T16:37:01.122119118Z Collecting performance related data for node ip-10-0-131-24.ec2.internal
2026-04-16T16:37:01.129525374Z Collecting performance related data for node ip-10-0-140-78.ec2.internal
2026-04-16T16:37:01.473385818Z Gathering data for ns/openshift-config...
2026-04-16T16:37:01.888551048Z Gathering data for ns/openshift-config-managed...
2026-04-16T16:37:02.510517709Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T16:37:02.875110324Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T16:37:03.055330862Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-04-16T16:37:03.177611560Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T16:37:03.572353048Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T16:37:03.957936553Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T16:37:04.305704982Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T16:37:04.659036822Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T16:37:05.012738549Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T16:37:05.402678395Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T16:37:06.018017741Z Gathering data for ns/openshift-multus...
2026-04-16T16:37:07.794595830Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T16:37:08.060485307Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-04-16T16:37:09.386465208Z Gathering data for ns/openshift-host-network...
2026-04-16T16:37:09.520262537Z Collecting kubelet logs for node ip-10-0-134-41.ec2.internal
2026-04-16T16:37:09.522137246Z Collecting kubelet logs for node ip-10-0-131-24.ec2.internal
2026-04-16T16:37:09.524084451Z Collecting kubelet logs for node ip-10-0-140-78.ec2.internal
2026-04-16T16:37:09.817397508Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T16:37:09.942823304Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T16:37:09.948854104Z INFO: Node performance data collection complete.
2026-04-16T16:37:10.399723454Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T16:37:10.938065964Z Gathering data for ns/openshift-network-operator...
2026-04-16T16:37:11.330585143Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T16:37:11.618751718Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T16:37:12.086685490Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T16:37:12.386825035Z Gathering data for ns/openshift-apiserver...
2026-04-16T16:37:12.821375930Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T16:37:13.067974032Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70
2026-04-16T16:37:13.133191295Z Gathering data for ns/openshift-controller-manager...
2026-04-16T16:37:13.447275939Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T16:37:14.080328870Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T16:37:14.412359243Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T16:37:14.747322409Z Gathering data for ns/openshift-service-ca...
2026-04-16T16:37:15.137427965Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T16:37:16.061050197Z Wrote inspect data to must-gather.
2026-04-16T16:37:16.061095588Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T16:37:16.061095588Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T16:37:16.193198103Z error: the server doesn't have a resource type "clusters"
2026-04-16T16:37:16.361270041Z Caches written to disk