2026-04-17T21:05:59.929456020Z [disk usage checker] Started
2026-04-17T21:05:59.960408334Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T21:06:00.154369474Z Gathering data for ns/openshift-cluster-version...
2026-04-17T21:06:00.320209826Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T21:06:00.409970840Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T21:06:00.509973227Z Gathering data for ns/default...
2026-04-17T21:06:00.837513696Z Gathering data for ns/openshift...
2026-04-17T21:06:01.279002404Z Gathering data for ns/kube-system...
2026-04-17T21:06:01.953775865Z Gathering data for ns/openshift-etcd...
2026-04-17T21:06:02.303649384Z Wrote inspect data to must-gather.
2026-04-17T21:06:02.303679746Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T21:06:02.303679746Z     namespaces "assisted-installer" not found
2026-04-17T21:06:02.374212584Z Waiting on subprocesses to finish execution.
2026-04-17T21:06:02.380200132Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-17T21:06:02.390714494Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-17T21:06:02.391680065Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-17T21:06:02.396516697Z INFO: Gathering HAProxy config files
2026-04-17T21:06:02.409792357Z INFO: Collecting host service logs for crio
2026-04-17T21:06:02.410146246Z INFO: Collecting host service logs for kubelet
2026-04-17T21:06:02.410479482Z INFO: Collecting host service logs for rpm-ostreed
2026-04-17T21:06:02.410825863Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-17T21:06:02.411196129Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-17T21:06:02.411537480Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-17T21:06:02.411944603Z INFO: Collecting host service logs for NetworkManager
2026-04-17T21:06:02.412357168Z INFO: Collecting host service logs for openvswitch
2026-04-17T21:06:02.412883691Z INFO: Collecting host service logs for ovs-configuration
2026-04-17T21:06:02.413214891Z INFO: Collecting host service logs for ovsdb-server
2026-04-17T21:06:02.413530916Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-17T21:06:02.414007425Z INFO: Waiting for worker host service log collection to complete ...
2026-04-17T21:06:02.626580318Z INFO: Waiting for node performance related collection to complete ...
2026-04-17T21:06:02.842741896Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-17T21:06:03.262637184Z Wrote inspect data to must-gather.
2026-04-17T21:06:03.271892494Z No resources found in openshift-etcd namespace.
2026-04-17T21:06:03.358703516Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-5jzh4
2026-04-17T21:06:03.358755119Z insights-runtime-extractor-nqw6n
2026-04-17T21:06:03.358772351Z insights-runtime-extractor-nxrn6
2026-04-17T21:06:03.358789500Z insights-runtime-extractor-rnnj4
2026-04-17T21:06:03.588150077Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-17T21:06:03.588272272Z INFO: on-disk MachineConfig config collection complete.
2026-04-17T21:06:03.598577804Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-17T21:06:03.775030159Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-17T21:06:03.775995457Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-17T21:06:03.804332406Z No resources found
2026-04-17T21:06:03.808779032Z error: the server doesn't have a resource type "performanceprofile"
2026-04-17T21:06:03.821735589Z INFO: Worker host service log collection to complete.
2026-04-17T21:06:03.852455901Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-17T21:06:03.885369782Z INFO: "metallb-operator" not detected. Skipping.
2026-04-17T21:06:04.040498027Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-17T21:06:04.040498027Z See 'oc rsync -h' for help and examples
2026-04-17T21:06:04.140214922Z INFO: HAProxy config collection complete.
2026-04-17T21:06:04.178764349Z ERROR: No running kube-apiserver pods found
2026-04-17T21:06:04.350361503Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-17T21:06:04.449533503Z Wrote inspect data to must-gather.
2026-04-17T21:06:04.507869463Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-17T21:06:04.885246077Z INFO: 'previous-logs' folder not found on ip-10-0-137-102.ec2.internal, skipping...
2026-04-17T21:06:04.890399169Z Wrote inspect data to must-gather.
2026-04-17T21:06:04.965111129Z Gathering data for ns/kuadrant-system...
2026-04-17T21:06:04.968963363Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T21:06:05.063255206Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T21:06:05.173281443Z No resources found
2026-04-17T21:06:05.193902968Z Wrote inspect data to must-gather.
2026-04-17T21:06:05.265202416Z error: the server doesn't have a resource type "machineconfigs"
2026-04-17T21:06:05.270863620Z Wrote inspect data to must-gather.
2026-04-17T21:06:05.333017538Z Wrote inspect data to must-gather.
2026-04-17T21:06:05.456485826Z INFO: 'previous-logs' folder not found on ip-10-0-137-110.ec2.internal, skipping...
2026-04-17T21:06:05.485409611Z INFO: Getting rules from prometheus-k8s-0
2026-04-17T21:06:05.614041189Z Wrote inspect data to must-gather.
2026-04-17T21:06:05.623077452Z Wrote inspect data to must-gather.
2026-04-17T21:06:05.832961409Z INFO: 'previous-logs' folder not found on ip-10-0-139-255.ec2.internal, skipping...
2026-04-17T21:06:05.833007017Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-17T21:06:05.833042284Z INFO: Machine Config Daemon termination log collection complete.
2026-04-17T21:06:05.950299377Z Wrote inspect data to must-gather.
2026-04-17T21:06:05.975311022Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T21:06:05.999924928Z Wrote inspect data to must-gather.
2026-04-17T21:06:06.022052557Z INFO: Getting status/config from prometheus-k8s-0
2026-04-17T21:06:06.186625607Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-17T21:06:06.390493134Z Wrote inspect data to must-gather.
2026-04-17T21:06:06.507894849Z W0417 21:06:06.507850    1118 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-17T21:06:06.536911738Z Wrote inspect data to must-gather.
2026-04-17T21:06:06.552242794Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-17T21:06:06.562177486Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T21:06:06.803408040Z INFO: INTERCONNECT MODE
2026-04-17T21:06:06.803495948Z INFO: Gathering ovn-kubernetes DBs
2026-04-17T21:06:06.844345577Z Gathering data for ns/openshift-monitoring...
2026-04-17T21:06:06.899829348Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-17T21:06:06.942673529Z Wrote inspect data to must-gather.
2026-04-17T21:06:07.012618062Z INFO: Gathering OVN_Northbound from ovnkube-node-dk97h...
2026-04-17T21:06:07.012950656Z INFO: Gathering OVN_Northbound from ovnkube-node-fdrzh...
2026-04-17T21:06:07.013235615Z INFO: Gathering OVN_Northbound from ovnkube-node-trqtd...
2026-04-17T21:06:07.013525960Z INFO: Gathering OVN_Southbound from ovnkube-node-dk97h...
2026-04-17T21:06:07.013785022Z INFO: Gathering OVN_Southbound from ovnkube-node-fdrzh...
2026-04-17T21:06:07.014075446Z INFO: Gathering OVN_Southbound from ovnkube-node-trqtd...
2026-04-17T21:06:07.188720936Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-17T21:06:07.375931318Z tar: Removing leading `/' from member names
2026-04-17T21:06:07.388187079Z tar: Removing leading `/' from member names
2026-04-17T21:06:07.404013286Z tar: Removing leading `/' from member names
2026-04-17T21:06:07.410408386Z tar: Removing leading `/' from member names
2026-04-17T21:06:07.413600666Z tar: Removing leading `/' from member names
2026-04-17T21:06:07.435193423Z Wrote inspect data to must-gather.
2026-04-17T21:06:07.456969756Z tar: Removing leading `/' from member names
2026-04-17T21:06:07.561215489Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-17T21:06:07.806605290Z Wrote inspect data to must-gather.
2026-04-17T21:06:08.041285565Z INFO: Getting status from alertmanager-main-0
2026-04-17T21:06:08.107170226Z INFO: Waiting for network log collection to complete ...
2026-04-17T21:06:08.107284698Z INFO: Waiting for ovnk database copies to complete ...
2026-04-17T21:06:08.107773285Z INFO: Copying ovnk databases complete.
2026-04-17T21:06:08.109936621Z 3.8M	must-gather/network_logs/ovnk_database_store
2026-04-17T21:06:08.112968256Z ovnk_database_store/
2026-04-17T21:06:08.113022837Z ovnk_database_store/ovnkube-node-trqtd_nbdb
2026-04-17T21:06:08.129472515Z ovnk_database_store/ovnkube-node-dk97h_nbdb
2026-04-17T21:06:08.141045423Z ovnk_database_store/ovnkube-node-dk97h_sbdb
2026-04-17T21:06:08.173829883Z ovnk_database_store/ovnkube-node-fdrzh_nbdb
2026-04-17T21:06:08.185378382Z ovnk_database_store/ovnkube-node-fdrzh_sbdb
2026-04-17T21:06:08.194418931Z Wrote inspect data to must-gather.
2026-04-17T21:06:08.220201174Z ovnk_database_store/ovnkube-node-trqtd_sbdb
2026-04-17T21:06:08.251962467Z INFO: Network log collection complete.
2026-04-17T21:06:08.547418021Z Wrote inspect data to must-gather.
2026-04-17T21:06:08.861584354Z Wrote inspect data to must-gather.
2026-04-17T21:06:09.166354273Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-17T21:06:09.169247793Z INFO: Fallback to identify the container image from release info
2026-04-17T21:06:09.207081793Z Gathering data for ns/openshift-network-console...
2026-04-17T21:06:09.682477943Z Gathering data for ns/openshift-console-operator...
2026-04-17T21:06:09.713338338Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-17T21:06:09.874202456Z daemonset.apps/perf-node-gather-daemonset created
2026-04-17T21:06:10.016062418Z Waiting for performance profile collector pods to become ready: 1
2026-04-17T21:06:10.070429186Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T21:06:10.183053740Z Gathering data for ns/openshift-console...
2026-04-17T21:06:10.740791609Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-17T21:06:11.162554482Z Waiting for performance profile collector pods to become ready: 2
2026-04-17T21:06:11.168779368Z Gathering data for ns/openshift-dns-operator...
2026-04-17T21:06:11.406843554Z Wrote inspect data to must-gather.
2026-04-17T21:06:11.496567854Z Gathering data for ns/openshift-dns...
2026-04-17T21:06:12.068322193Z Gathering data for ns/openshift-image-registry...
2026-04-17T21:06:12.299921761Z Waiting for performance profile collector pods to become ready: 3
2026-04-17T21:06:12.548594228Z Gathering data for ns/openshift-ingress-operator...
2026-04-17T21:06:12.880350178Z Gathering data for ns/openshift-ingress...
2026-04-17T21:06:13.421929076Z Waiting for performance profile collector pods to become ready: 4
2026-04-17T21:06:13.487493288Z Gathering data for ns/openshift-ingress-canary...
2026-04-17T21:06:13.910424087Z Gathering data for ns/openshift-insights...
2026-04-17T21:06:14.551827427Z Waiting for performance profile collector pods to become ready: 5
2026-04-17T21:06:15.075085167Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T21:06:15.663347816Z Waiting for performance profile collector pods to become ready: 6
2026-04-17T21:06:15.868079963Z Gathering data for ns/opendatahub...
2026-04-17T21:06:16.770917596Z Waiting for performance profile collector pods to become ready: 7
2026-04-17T21:06:17.054559648Z Gathering data for ns/openshift-lws-operator...
2026-04-17T21:06:17.888532656Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-17T21:06:18.132072463Z Collecting performance related data for node ip-10-0-139-255.ec2.internal
2026-04-17T21:06:18.139957416Z Collecting performance related data for node ip-10-0-137-110.ec2.internal
2026-04-17T21:06:18.149361865Z Collecting performance related data for node ip-10-0-137-102.ec2.internal
2026-04-17T21:06:18.321605081Z Gathering data for ns/openshift-config...
2026-04-17T21:06:18.675512200Z Gathering data for ns/openshift-config-managed...
2026-04-17T21:06:19.301237334Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-17T21:06:19.687820556Z Gathering data for ns/openshift-kube-apiserver...
2026-04-17T21:06:19.998486086Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-17T21:06:20.084504907Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T21:06:20.324099343Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-17T21:06:20.696913053Z Gathering data for ns/openshift-kube-scheduler...
2026-04-17T21:06:21.018749824Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-17T21:06:21.361558499Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-17T21:06:21.711948469Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-17T21:06:22.040034005Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-17T21:06:22.584381986Z Gathering data for ns/openshift-multus...
2026-04-17T21:06:23.978017442Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-17T21:06:25.091133347Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T21:06:25.456622770Z Gathering data for ns/openshift-host-network...
2026-04-17T21:06:25.810897106Z Gathering data for ns/openshift-network-diagnostics...
2026-04-17T21:06:26.357707877Z Gathering data for ns/openshift-network-node-identity...
2026-04-17T21:06:26.799951014Z Gathering data for ns/openshift-network-operator...
2026-04-17T21:06:26.812582707Z Collecting kubelet logs for node ip-10-0-139-255.ec2.internal
2026-04-17T21:06:26.814454374Z Collecting kubelet logs for node ip-10-0-137-110.ec2.internal
2026-04-17T21:06:26.816203033Z Collecting kubelet logs for node ip-10-0-137-102.ec2.internal
2026-04-17T21:06:27.171987603Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-17T21:06:27.177972592Z INFO: Node performance data collection complete.
2026-04-17T21:06:27.272036385Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-17T21:06:27.576520374Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-17T21:06:28.026650649Z Gathering data for ns/openshift-apiserver-operator...
2026-04-17T21:06:28.333294355Z Gathering data for ns/openshift-apiserver...
2026-04-17T21:06:28.770294178Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-17T21:06:29.077661312Z Gathering data for ns/openshift-controller-manager...
2026-04-17T21:06:29.379854154Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-17T21:06:30.021852391Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-17T21:06:30.098710127Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-17T21:06:30.343329577Z Gathering data for ns/openshift-service-ca-operator...
2026-04-17T21:06:30.708133630Z Gathering data for ns/openshift-service-ca...
2026-04-17T21:06:31.081997387Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-17T21:06:32.017278377Z Wrote inspect data to must-gather.
2026-04-17T21:06:32.017321117Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T21:06:32.017321117Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-17T21:06:32.147461295Z error: the server doesn't have a resource type "clusters"
2026-04-17T21:06:32.333474381Z Caches written to disk