2026-04-20T15:29:27.546533128Z [disk usage checker] Started
2026-04-20T15:29:27.579278432Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-20T15:29:27.794641881Z Gathering data for ns/openshift-cluster-version...
2026-04-20T15:29:27.947028608Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T15:29:28.035053509Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T15:29:28.138752440Z Gathering data for ns/default...
2026-04-20T15:29:28.468120970Z Gathering data for ns/openshift...
2026-04-20T15:29:28.935176500Z Gathering data for ns/kube-system...
2026-04-20T15:29:29.651484817Z Gathering data for ns/openshift-etcd...
2026-04-20T15:29:29.999789954Z Wrote inspect data to must-gather.
2026-04-20T15:29:29.999825447Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T15:29:29.999825447Z     namespaces "assisted-installer" not found
2026-04-20T15:29:30.049544657Z Waiting on subprocesses to finish execution.
2026-04-20T15:29:30.054343600Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-20T15:29:30.069110326Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-20T15:29:30.073581125Z INFO: Collecting host service logs for crio
2026-04-20T15:29:30.073789378Z INFO: Gathering HAProxy config files
2026-04-20T15:29:30.073996029Z INFO: Collecting host service logs for kubelet
2026-04-20T15:29:30.074308383Z INFO: Collecting host service logs for rpm-ostreed
2026-04-20T15:29:30.074603845Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-20T15:29:30.074909938Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-20T15:29:30.075189838Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-20T15:29:30.075500716Z INFO: Collecting host service logs for NetworkManager
2026-04-20T15:29:30.075869389Z INFO: Collecting host service logs for openvswitch
2026-04-20T15:29:30.076304385Z INFO: Collecting host service logs for ovs-configuration
2026-04-20T15:29:30.076644015Z INFO: Collecting host service logs for ovsdb-server
2026-04-20T15:29:30.077013158Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-20T15:29:30.077426174Z INFO: Waiting for worker host service log collection to complete ...
2026-04-20T15:29:30.086480056Z INFO: Waiting for node performance related collection to complete ...
2026-04-20T15:29:30.141800267Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-20T15:29:30.821311647Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-20T15:29:30.896116213Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-20T15:29:30.905903747Z No resources found in openshift-etcd namespace.
2026-04-20T15:29:30.905933953Z INFO: "metallb-operator" not detected. Skipping.
2026-04-20T15:29:31.016784838Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-20T15:29:31.102424329Z No resources found
2026-04-20T15:29:31.135396614Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-xc2s2
2026-04-20T15:29:31.135444468Z insights-runtime-extractor-fsfq2
2026-04-20T15:29:31.135465066Z insights-runtime-extractor-gktqt
2026-04-20T15:29:31.135485293Z insights-runtime-extractor-qsjv2
2026-04-20T15:29:31.789353006Z No resources found
2026-04-20T15:29:31.919057660Z error: the server doesn't have a resource type "performanceprofile"
2026-04-20T15:29:31.930015924Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-20T15:29:31.930101542Z INFO: on-disk MachineConfig config collection complete.
2026-04-20T15:29:31.935418275Z Wrote inspect data to must-gather.
2026-04-20T15:29:31.952401146Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T15:29:31.996548754Z Wrote inspect data to must-gather.
2026-04-20T15:29:32.124814030Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-20T15:29:32.136120028Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-20T15:29:32.136120028Z See 'oc rsync -h' for help and examples
2026-04-20T15:29:32.137717700Z INFO: Worker host service log collection to complete.
2026-04-20T15:29:32.163352533Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-20T15:29:32.188059175Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-20T15:29:32.269605341Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-20T15:29:32.354863306Z Wrote inspect data to must-gather.
2026-04-20T15:29:32.446102588Z Wrote inspect data to must-gather.
2026-04-20T15:29:32.612329756Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-20T15:29:32.637128205Z tar: Removing leading `/' from member names
2026-04-20T15:29:32.644493978Z INFO: HAProxy config collection complete.
2026-04-20T15:29:32.647998452Z Wrote inspect data to must-gather.
2026-04-20T15:29:32.690778416Z ERROR: No running kube-apiserver pods found
2026-04-20T15:29:32.877204462Z Wrote inspect data to must-gather.
2026-04-20T15:29:32.903654262Z INFO: 'previous-logs' folder not found on ip-10-0-130-249.ec2.internal, skipping...
2026-04-20T15:29:33.020516800Z Gathering data for ns/kuadrant-system...
2026-04-20T15:29:33.038771328Z Wrote inspect data to must-gather.
2026-04-20T15:29:33.107157292Z error: the server doesn't have a resource type "machineconfigs"
2026-04-20T15:29:33.144670731Z INFO: Getting rules from prometheus-k8s-0
2026-04-20T15:29:33.289117562Z INFO: 'previous-logs' folder not found on ip-10-0-133-163.ec2.internal, skipping...
2026-04-20T15:29:33.411668951Z Wrote inspect data to must-gather.
2026-04-20T15:29:33.429391389Z Wrote inspect data to must-gather.
2026-04-20T15:29:33.625589806Z INFO: 'previous-logs' folder not found on ip-10-0-141-9.ec2.internal, skipping...
2026-04-20T15:29:33.625721933Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-20T15:29:33.625779027Z INFO: Machine Config Daemon termination log collection complete.
2026-04-20T15:29:33.701083917Z INFO: Getting status/config from prometheus-k8s-0
2026-04-20T15:29:33.789598645Z Wrote inspect data to must-gather.
2026-04-20T15:29:33.822711036Z Wrote inspect data to must-gather.
2026-04-20T15:29:34.033194140Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-20T15:29:34.110755059Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T15:29:34.206176515Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-20T15:29:34.218223349Z INFO: INTERCONNECT MODE
2026-04-20T15:29:34.218316472Z INFO: Gathering ovn-kubernetes DBs
2026-04-20T15:29:34.334009180Z Wrote inspect data to must-gather.
2026-04-20T15:29:34.458157615Z INFO: Gathering OVN_Northbound from ovnkube-node-5vmgd...
2026-04-20T15:29:34.458547345Z INFO: Gathering OVN_Northbound from ovnkube-node-g9x87...
2026-04-20T15:29:34.458952215Z INFO: Gathering OVN_Northbound from ovnkube-node-qpw9h...
2026-04-20T15:29:34.459271945Z INFO: Gathering OVN_Southbound from ovnkube-node-5vmgd...
2026-04-20T15:29:34.459585935Z INFO: Gathering OVN_Southbound from ovnkube-node-g9x87...
2026-04-20T15:29:34.459918708Z INFO: Gathering OVN_Southbound from ovnkube-node-qpw9h...
2026-04-20T15:29:34.504112318Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-20T15:29:34.736667699Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T15:29:34.785209609Z tar: Removing leading `/' from member names
2026-04-20T15:29:34.814896484Z tar: Removing leading `/' from member names
2026-04-20T15:29:34.841870265Z tar: Removing leading `/' from member names
2026-04-20T15:29:34.912546245Z Wrote inspect data to must-gather.
2026-04-20T15:29:34.960581446Z tar: Removing leading `/' from member names
2026-04-20T15:29:34.971796698Z tar: Removing leading `/' from member names
2026-04-20T15:29:34.996996980Z tar: Removing leading `/' from member names
2026-04-20T15:29:35.054111505Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-20T15:29:35.072479888Z Gathering data for ns/openshift-monitoring...
2026-04-20T15:29:35.309851939Z Wrote inspect data to must-gather.
2026-04-20T15:29:35.352949621Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-20T15:29:35.667537279Z INFO: Waiting for network log collection to complete ...
2026-04-20T15:29:35.667574553Z INFO: Waiting for ovnk database copies to complete ...
2026-04-20T15:29:35.668099213Z INFO: Copying ovnk databases complete.
2026-04-20T15:29:35.673316860Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-04-20T15:29:35.676141543Z ovnk_database_store/
2026-04-20T15:29:35.676141543Z ovnk_database_store/ovnkube-node-g9x87_sbdb
2026-04-20T15:29:35.703041387Z ovnk_database_store/ovnkube-node-5vmgd_nbdb
2026-04-20T15:29:35.705211020Z Wrote inspect data to must-gather.
2026-04-20T15:29:35.715342880Z ovnk_database_store/ovnkube-node-5vmgd_sbdb
2026-04-20T15:29:35.742545290Z ovnk_database_store/ovnkube-node-g9x87_nbdb
2026-04-20T15:29:35.751989936Z ovnk_database_store/ovnkube-node-qpw9h_nbdb
2026-04-20T15:29:35.766154436Z ovnk_database_store/ovnkube-node-qpw9h_sbdb
2026-04-20T15:29:35.791427278Z INFO: Network log collection complete.
2026-04-20T15:29:35.801314193Z INFO: Getting status from alertmanager-main-0
2026-04-20T15:29:36.037103043Z Wrote inspect data to must-gather.
2026-04-20T15:29:36.365897103Z Wrote inspect data to must-gather.
2026-04-20T15:29:36.700146551Z Wrote inspect data to must-gather.
2026-04-20T15:29:36.972870210Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-20T15:29:36.975238635Z INFO: Fallback to identify the container image from release info
2026-04-20T15:29:37.246577442Z Gathering data for ns/openshift-network-console...
2026-04-20T15:29:37.623492553Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-20T15:29:37.687089813Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-20T15:29:37.733645173Z Gathering data for ns/openshift-console-operator...
2026-04-20T15:29:37.819892782Z daemonset.apps/perf-node-gather-daemonset created
2026-04-20T15:29:37.984345224Z Waiting for performance profile collector pods to become ready: 1
2026-04-20T15:29:38.193108572Z Gathering data for ns/openshift-console...
2026-04-20T15:29:38.741634195Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-20T15:29:39.134272170Z Waiting for performance profile collector pods to become ready: 2
2026-04-20T15:29:39.167085759Z Gathering data for ns/openshift-dns-operator...
2026-04-20T15:29:39.513389609Z Gathering data for ns/openshift-dns...
2026-04-20T15:29:39.632579846Z Wrote inspect data to must-gather.
2026-04-20T15:29:40.098366269Z Gathering data for ns/openshift-image-registry...
2026-04-20T15:29:40.251830391Z Waiting for performance profile collector pods to become ready: 3
2026-04-20T15:29:40.571489939Z Gathering data for ns/openshift-ingress-operator...
2026-04-20T15:29:40.939980440Z Gathering data for ns/openshift-ingress...
2026-04-20T15:29:41.386244715Z Waiting for performance profile collector pods to become ready: 4
2026-04-20T15:29:41.804405971Z Gathering data for ns/openshift-ingress-canary...
2026-04-20T15:29:42.246676352Z Gathering data for ns/openshift-insights...
2026-04-20T15:29:42.502005896Z Waiting for performance profile collector pods to become ready: 5
2026-04-20T15:29:42.629318056Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-20T15:29:43.624512983Z Waiting for performance profile collector pods to become ready: 6
2026-04-20T15:29:44.267189252Z Gathering data for ns/opendatahub...
2026-04-20T15:29:44.762758619Z Waiting for performance profile collector pods to become ready: 7
2026-04-20T15:29:45.829172513Z Gathering data for ns/openshift-lws-operator...
2026-04-20T15:29:45.879632008Z Waiting for performance profile collector pods to become ready: 8
2026-04-20T15:29:47.008018091Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-20T15:29:47.010712471Z Gathering data for ns/openshift-config...
2026-04-20T15:29:47.287272152Z Collecting performance related data for node ip-10-0-141-9.ec2.internal
2026-04-20T15:29:47.297023623Z Collecting performance related data for node ip-10-0-133-163.ec2.internal
2026-04-20T15:29:47.311900044Z Collecting performance related data for node ip-10-0-130-249.ec2.internal
2026-04-20T15:29:47.438581714Z Gathering data for ns/openshift-config-managed...
2026-04-20T15:29:47.636818868Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-20T15:29:48.008001641Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-20T15:29:48.404446994Z Gathering data for ns/openshift-kube-apiserver...
2026-04-20T15:29:48.770329401Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-20T15:29:49.145022157Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-20T15:29:49.454223743Z Gathering data for ns/openshift-kube-scheduler...
2026-04-20T15:29:49.788139772Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-20T15:29:50.180336565Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-20T15:29:50.572243455Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-20T15:29:50.894922699Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-20T15:29:51.431222418Z Gathering data for ns/openshift-multus...
2026-04-20T15:29:52.643895218Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-20T15:29:52.828171275Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-20T15:29:54.345959573Z Gathering data for ns/openshift-host-network...
2026-04-20T15:29:54.678379221Z Gathering data for ns/openshift-network-diagnostics...
2026-04-20T15:29:55.326854093Z Gathering data for ns/openshift-network-node-identity...
2026-04-20T15:29:55.795536504Z Gathering data for ns/openshift-network-operator...
2026-04-20T15:29:55.795701034Z Collecting kubelet logs for node ip-10-0-141-9.ec2.internal
2026-04-20T15:29:55.797358008Z Collecting kubelet logs for node ip-10-0-133-163.ec2.internal
2026-04-20T15:29:55.799153922Z Collecting kubelet logs for node ip-10-0-130-249.ec2.internal
2026-04-20T15:29:56.161131436Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-20T15:29:56.166455966Z INFO: Node performance data collection complete.
2026-04-20T15:29:56.262285709Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-20T15:29:56.565159691Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-20T15:29:57.034950065Z Gathering data for ns/openshift-apiserver-operator...
2026-04-20T15:29:57.375938404Z Gathering data for ns/openshift-apiserver...
2026-04-20T15:29:57.649911263Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-20T15:29:57.818205214Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-20T15:29:58.097229201Z Gathering data for ns/openshift-controller-manager...
2026-04-20T15:29:58.407788303Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-20T15:29:59.095851475Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-20T15:29:59.425238845Z Gathering data for ns/openshift-service-ca-operator...
2026-04-20T15:29:59.748792172Z Gathering data for ns/openshift-service-ca...
2026-04-20T15:30:00.142815195Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-20T15:30:01.146604278Z Wrote inspect data to must-gather.
2026-04-20T15:30:01.146647526Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T15:30:01.146647526Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-20T15:30:01.278094002Z error: the server doesn't have a resource type "clusters"
2026-04-20T15:30:01.479664628Z Caches written to disk