2026-04-22T21:44:15.728069750Z [disk usage checker] Started
2026-04-22T21:44:15.762870563Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-22T21:44:15.951500668Z Gathering data for ns/openshift-cluster-version...
2026-04-22T21:44:16.103560823Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T21:44:16.195375823Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T21:44:16.302369133Z Gathering data for ns/default...
2026-04-22T21:44:16.664915669Z Gathering data for ns/openshift...
2026-04-22T21:44:17.149552593Z Gathering data for ns/kube-system...
2026-04-22T21:44:17.879724518Z Gathering data for ns/openshift-etcd...
2026-04-22T21:44:18.222015500Z Wrote inspect data to must-gather.
2026-04-22T21:44:18.222057739Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T21:44:18.222057739Z     namespaces "assisted-installer" not found
2026-04-22T21:44:18.274977270Z Waiting on subprocesses to finish execution.
2026-04-22T21:44:18.281062105Z INFO: Gathering HAProxy config files
2026-04-22T21:44:18.282876228Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T21:44:18.291576677Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T21:44:18.293562037Z INFO: Collecting host service logs for crio
2026-04-22T21:44:18.293664810Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T21:44:18.293872611Z INFO: Collecting host service logs for kubelet
2026-04-22T21:44:18.294260429Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T21:44:18.294590591Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T21:44:18.294939142Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T21:44:18.295293750Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T21:44:18.295659888Z INFO: Collecting host service logs for NetworkManager
2026-04-22T21:44:18.295992226Z INFO: Collecting host service logs for openvswitch
2026-04-22T21:44:18.296323907Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T21:44:18.296762773Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T21:44:18.297131510Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T21:44:18.297635191Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T21:44:18.358730125Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T21:44:18.724110945Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-gwzzw
2026-04-22T21:44:18.724165667Z insights-runtime-extractor-5d2jm
2026-04-22T21:44:18.724185233Z insights-runtime-extractor-kdgqq
2026-04-22T21:44:18.724204286Z insights-runtime-extractor-ldx4t
2026-04-22T21:44:18.893132651Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T21:44:18.948492009Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T21:44:18.948581150Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T21:44:19.169853228Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T21:44:19.222447810Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T21:44:19.327143189Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T21:44:19.367332517Z Wrote inspect data to must-gather.
2026-04-22T21:44:19.467330545Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T21:44:19.470803147Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T21:44:19.470803147Z See 'oc rsync -h' for help and examples
2026-04-22T21:44:19.484337318Z No resources found
2026-04-22T21:44:19.577518874Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T21:44:19.707291111Z No resources found in openshift-etcd namespace.
2026-04-22T21:44:19.795424962Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T21:44:19.905665908Z ERROR: No running kube-apiserver pods found
2026-04-22T21:44:20.021313405Z INFO: Worker host service log collection to complete.
2026-04-22T21:44:20.605076238Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T21:44:20.680432992Z Wrote inspect data to must-gather.
2026-04-22T21:44:20.735018967Z Wrote inspect data to must-gather.
2026-04-22T21:44:20.794319861Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T21:44:20.803852943Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-22T21:44:20.808114784Z INFO: 'previous-logs' folder not found on ip-10-0-134-137.ec2.internal, skipping...
2026-04-22T21:44:20.814707747Z Wrote inspect data to must-gather.
2026-04-22T21:44:20.844210634Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T21:44:20.882515488Z Wrote inspect data to must-gather.
2026-04-22T21:44:20.897026073Z No resources found
2026-04-22T21:44:21.092118040Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T21:44:21.095195704Z Gathering data for ns/kuadrant-system...
2026-04-22T21:44:21.136700708Z tar: Removing leading `/' from member names
2026-04-22T21:44:21.142996919Z INFO: HAProxy config collection complete.
2026-04-22T21:44:21.213269144Z Wrote inspect data to must-gather.
2026-04-22T21:44:21.257269400Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T21:44:21.271797986Z INFO: 'previous-logs' folder not found on ip-10-0-136-20.ec2.internal, skipping...
2026-04-22T21:44:21.324728485Z Wrote inspect data to must-gather.
2026-04-22T21:44:21.632772319Z Wrote inspect data to must-gather.
2026-04-22T21:44:21.674548217Z INFO: 'previous-logs' folder not found on ip-10-0-143-252.ec2.internal, skipping...
2026-04-22T21:44:21.674585133Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T21:44:21.674621846Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T21:44:21.683963595Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T21:44:21.805275625Z Wrote inspect data to must-gather.
2026-04-22T21:44:21.981223220Z Wrote inspect data to must-gather.
2026-04-22T21:44:22.155834252Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T21:44:22.198505120Z Wrote inspect data to must-gather.
2026-04-22T21:44:22.217596539Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T21:44:22.239309493Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T21:44:22.497200694Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T21:44:22.538263847Z Wrote inspect data to must-gather.
2026-04-22T21:44:22.612495547Z INFO: INTERCONNECT MODE
2026-04-22T21:44:22.612595017Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T21:44:22.736891264Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T21:44:22.759972264Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T21:44:22.823838140Z INFO: Gathering OVN_Northbound from ovnkube-node-8ngls...
2026-04-22T21:44:22.824166734Z INFO: Gathering OVN_Northbound from ovnkube-node-j6pw2...
2026-04-22T21:44:22.824679719Z INFO: Gathering OVN_Northbound from ovnkube-node-wqvs8...
2026-04-22T21:44:22.824988280Z INFO: Gathering OVN_Southbound from ovnkube-node-8ngls...
2026-04-22T21:44:22.825240573Z INFO: Gathering OVN_Southbound from ovnkube-node-j6pw2...
2026-04-22T21:44:22.825529964Z INFO: Gathering OVN_Southbound from ovnkube-node-wqvs8...
2026-04-22T21:44:23.042626363Z Wrote inspect data to must-gather.
2026-04-22T21:44:23.082591306Z Gathering data for ns/openshift-monitoring...
2026-04-22T21:44:23.169925506Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T21:44:23.253801050Z tar: Removing leading `/' from member names
2026-04-22T21:44:23.270614555Z tar: Removing leading `/' from member names
2026-04-22T21:44:23.311317862Z tar: Removing leading `/' from member names
2026-04-22T21:44:23.333320474Z tar: Removing leading `/' from member names
2026-04-22T21:44:23.350252166Z tar: Removing leading `/' from member names
2026-04-22T21:44:23.351972861Z tar: Removing leading `/' from member names
2026-04-22T21:44:23.602913381Z Wrote inspect data to must-gather.
2026-04-22T21:44:23.736118008Z INFO: Getting status from alertmanager-main-0
2026-04-22T21:44:23.964068868Z INFO: Waiting for network log collection to complete ...
2026-04-22T21:44:23.964098950Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T21:44:23.964511128Z INFO: Copying ovnk databases complete.
2026-04-22T21:44:23.966105352Z 4.1M	must-gather/network_logs/ovnk_database_store
2026-04-22T21:44:23.968343365Z ovnk_database_store/
2026-04-22T21:44:23.968343365Z ovnk_database_store/ovnkube-node-wqvs8_sbdb
2026-04-22T21:44:23.994164208Z Wrote inspect data to must-gather.
2026-04-22T21:44:23.996266269Z ovnk_database_store/ovnkube-node-8ngls_sbdb
2026-04-22T21:44:24.023480744Z ovnk_database_store/ovnkube-node-wqvs8_nbdb
2026-04-22T21:44:24.034455483Z ovnk_database_store/ovnkube-node-j6pw2_sbdb
2026-04-22T21:44:24.060958385Z ovnk_database_store/ovnkube-node-8ngls_nbdb
2026-04-22T21:44:24.073702445Z ovnk_database_store/ovnkube-node-j6pw2_nbdb
2026-04-22T21:44:24.089610956Z INFO: Network log collection complete.
2026-04-22T21:44:24.322227327Z Wrote inspect data to must-gather.
2026-04-22T21:44:24.691686116Z Wrote inspect data to must-gather.
2026-04-22T21:44:25.036423761Z Wrote inspect data to must-gather.
2026-04-22T21:44:25.346243827Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T21:44:25.349882114Z INFO: Fallback to identify the container image from release info
2026-04-22T21:44:25.523097668Z Gathering data for ns/openshift-network-console...
2026-04-22T21:44:25.824602355Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-22T21:44:25.933543884Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T21:44:26.037031118Z Gathering data for ns/openshift-console-operator...
2026-04-22T21:44:26.079000232Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T21:44:26.243173962Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T21:44:26.542093942Z Gathering data for ns/openshift-console...
2026-04-22T21:44:27.038350411Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T21:44:27.403429304Z Waiting for performance profile collector pods to become ready: 2
2026-04-22T21:44:27.489485851Z Gathering data for ns/openshift-dns-operator...
2026-04-22T21:44:27.869853330Z Gathering data for ns/openshift-dns...
2026-04-22T21:44:28.167072075Z Wrote inspect data to must-gather.
2026-04-22T21:44:28.462362098Z Gathering data for ns/openshift-image-registry...
2026-04-22T21:44:28.539665897Z Waiting for performance profile collector pods to become ready: 3
2026-04-22T21:44:28.914470991Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T21:44:29.242117512Z Gathering data for ns/openshift-ingress...
2026-04-22T21:44:29.675999767Z Waiting for performance profile collector pods to become ready: 4
2026-04-22T21:44:30.017093654Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T21:44:30.499525888Z Gathering data for ns/openshift-insights...
2026-04-22T21:44:30.812923249Z Waiting for performance profile collector pods to become ready: 5
2026-04-22T21:44:30.829690682Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-22T21:44:31.924077169Z Waiting for performance profile collector pods to become ready: 6
2026-04-22T21:44:32.459196908Z Gathering data for ns/opendatahub...
2026-04-22T21:44:33.038613056Z Waiting for performance profile collector pods to become ready: 7
2026-04-22T21:44:33.904877832Z Gathering data for ns/openshift-lws-operator...
2026-04-22T21:44:34.162023724Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T21:44:34.442557673Z Collecting performance related data for node ip-10-0-143-252.ec2.internal
2026-04-22T21:44:34.449539414Z Collecting performance related data for node ip-10-0-134-137.ec2.internal
2026-04-22T21:44:34.457367431Z Collecting performance related data for node ip-10-0-136-20.ec2.internal
2026-04-22T21:44:35.326378150Z Gathering data for ns/openshift-config...
2026-04-22T21:44:35.726856341Z Gathering data for ns/openshift-config-managed...
2026-04-22T21:44:35.843930481Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-22T21:44:36.215319832Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T21:44:36.527715568Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T21:44:36.841206254Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T21:44:37.229297201Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T21:44:37.579163722Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T21:44:37.886440545Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T21:44:38.216735520Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T21:44:38.604971182Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T21:44:38.932544582Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T21:44:39.499493863Z Gathering data for ns/openshift-multus...
2026-04-22T21:44:40.807362302Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T21:44:40.853051852Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-22T21:44:42.271563102Z Gathering data for ns/openshift-host-network...
2026-04-22T21:44:42.616178959Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T21:44:43.090025549Z Collecting kubelet logs for node ip-10-0-143-252.ec2.internal
2026-04-22T21:44:43.091798212Z Collecting kubelet logs for node ip-10-0-134-137.ec2.internal
2026-04-22T21:44:43.094999882Z Collecting kubelet logs for node ip-10-0-136-20.ec2.internal
2026-04-22T21:44:43.188718187Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T21:44:43.513998614Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T21:44:43.520685962Z INFO: Node performance data collection complete.
2026-04-22T21:44:43.730302377Z Gathering data for ns/openshift-network-operator...
2026-04-22T21:44:44.232515376Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T21:44:44.515260693Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T21:44:44.976133359Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T21:44:45.276268950Z Gathering data for ns/openshift-apiserver...
2026-04-22T21:44:45.708999588Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T21:44:45.857601831Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-22T21:44:46.024147690Z Gathering data for ns/openshift-controller-manager...
2026-04-22T21:44:46.311125139Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T21:44:47.017892892Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T21:44:47.368317603Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T21:44:47.702289578Z Gathering data for ns/openshift-service-ca...
2026-04-22T21:44:48.067067600Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T21:44:49.027317487Z Wrote inspect data to must-gather.
2026-04-22T21:44:49.027361459Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T21:44:49.027361459Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T21:44:49.163306987Z error: the server doesn't have a resource type "clusters"
2026-04-22T21:44:49.355891795Z Caches written to disk