2026-06-08T12:11:43.685694631Z [disk usage checker] Started
2026-06-08T12:11:43.689589944Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-08T12:11:43.884620042Z Gathering data for ns/openshift-cluster-version...
2026-06-08T12:11:44.008492859Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T12:11:44.093838135Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T12:11:44.172315755Z Gathering data for ns/default...
2026-06-08T12:11:44.499380634Z Gathering data for ns/openshift...
2026-06-08T12:11:44.914228877Z Gathering data for ns/kube-system...
2026-06-08T12:11:45.903531233Z Gathering data for ns/openshift-etcd...
2026-06-08T12:11:46.194060065Z Wrote inspect data to must-gather.
2026-06-08T12:11:46.194102855Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T12:11:46.194102855Z     namespaces "assisted-installer" not found
2026-06-08T12:11:46.213552603Z Waiting on subprocesses to finish execution.
2026-06-08T12:11:46.219318906Z Executing Istio gather script
2026-06-08T12:11:46.225204738Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-08T12:11:46.230292598Z INFO: Collecting host service logs for crio
2026-06-08T12:11:46.230549582Z INFO: Collecting host service logs for kubelet
2026-06-08T12:11:46.230841022Z INFO: Collecting host service logs for rpm-ostreed
2026-06-08T12:11:46.231317563Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-08T12:11:46.231626156Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-08T12:11:46.231957384Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-08T12:11:46.232254570Z INFO: Collecting host service logs for NetworkManager
2026-06-08T12:11:46.232564766Z INFO: Collecting host service logs for openvswitch
2026-06-08T12:11:46.232984206Z INFO: Collecting host service logs for ovs-configuration
2026-06-08T12:11:46.233331153Z INFO: Collecting host service logs for ovsdb-server
2026-06-08T12:11:46.233621671Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-08T12:11:46.234046062Z INFO: Waiting for worker host service log collection to complete ...
2026-06-08T12:11:46.235661495Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-08T12:11:46.239476094Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-08T12:11:46.241542038Z INFO: Gathering HAProxy config files
2026-06-08T12:11:46.248763319Z INFO: Waiting for node performance related collection to complete ...
2026-06-08T12:11:46.762446492Z INFO: Collecting Insights Archives from insights-operator-896549976-4bxmm
2026-06-08T12:11:46.762514115Z insights-runtime-extractor-jg922
2026-06-08T12:11:46.762536100Z insights-runtime-extractor-lggnf
2026-06-08T12:11:46.762556697Z insights-runtime-extractor-w5k7f
2026-06-08T12:11:46.834334442Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-08T12:11:46.986644375Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-08T12:11:47.142961821Z error: the server doesn't have a resource type "performanceprofile"
2026-06-08T12:11:47.184198127Z Wrote inspect data to must-gather.
2026-06-08T12:11:47.236163317Z ERROR: No running kube-apiserver pods found
2026-06-08T12:11:47.276549624Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-08T12:11:47.276631853Z INFO: on-disk MachineConfig config collection complete.
2026-06-08T12:11:47.303146396Z INFO: "metallb-operator" not detected. Skipping.
2026-06-08T12:11:47.306422612Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-08T12:11:47.346681557Z No resources found
2026-06-08T12:11:47.473677344Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-08T12:11:47.537588224Z No resources found in openshift-etcd namespace.
2026-06-08T12:11:47.544713210Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-08T12:11:47.620051798Z INFO: Worker host service log collection to complete.
2026-06-08T12:11:47.656205946Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-08T12:11:47.656205946Z See 'oc rsync -h' for help and examples
2026-06-08T12:11:47.959592321Z Wrote inspect data to must-gather.
2026-06-08T12:11:48.007351928Z INFO: 'previous-logs' folder not found on ip-10-0-132-76.ec2.internal, skipping...
2026-06-08T12:11:48.021247170Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-08T12:11:48.091500146Z Wrote inspect data to must-gather.
2026-06-08T12:11:48.352321357Z Gathering data for ns/openshift-monitoring...
2026-06-08T12:11:48.436503271Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-08T12:11:48.631407396Z No resources found
2026-06-08T12:11:48.647660492Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T12:11:48.691949833Z error: the server doesn't have a resource type "machineconfigs"
2026-06-08T12:11:48.695593074Z Wrote inspect data to must-gather.
2026-06-08T12:11:48.737956642Z Wrote inspect data to must-gather.
2026-06-08T12:11:48.740129273Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-08T12:11:48.741738584Z INFO: Getting rules from prometheus-k8s-0
2026-06-08T12:11:48.796735120Z Wrote inspect data to must-gather.
2026-06-08T12:11:48.821972970Z INFO: 'previous-logs' folder not found on ip-10-0-135-253.ec2.internal, skipping...
2026-06-08T12:11:48.868152096Z Wrote inspect data to must-gather.
2026-06-08T12:11:48.868183073Z tar: Removing leading `/' from member names
2026-06-08T12:11:48.876238935Z INFO: HAProxy config collection complete.
2026-06-08T12:11:48.945028876Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-08T12:11:49.028209322Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-08T12:11:49.089946048Z Wrote inspect data to must-gather.
2026-06-08T12:11:49.105991218Z INFO: 'previous-logs' folder not found on ip-10-0-137-95.ec2.internal, skipping...
2026-06-08T12:11:49.106025924Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-08T12:11:49.106025924Z INFO: Machine Config Daemon termination log collection complete.
2026-06-08T12:11:49.189768863Z Wrote inspect data to must-gather.
2026-06-08T12:11:49.215126936Z INFO: Getting status/config from prometheus-k8s-0
2026-06-08T12:11:49.342285929Z Wrote inspect data to must-gather.
2026-06-08T12:11:49.350683215Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-08T12:11:49.545479391Z Wrote inspect data to must-gather.
2026-06-08T12:11:49.558742863Z Wrote inspect data to must-gather.
2026-06-08T12:11:49.622943284Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-08T12:11:49.742283868Z Wrote inspect data to must-gather.
2026-06-08T12:11:49.827405397Z INFO: INTERCONNECT MODE
2026-06-08T12:11:49.827405397Z INFO: Gathering ovn-kubernetes DBs
2026-06-08T12:11:49.866987434Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-08T12:11:49.894305020Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-08T12:11:49.945960022Z Wrote inspect data to must-gather.
2026-06-08T12:11:49.993072316Z INFO: Gathering OVN_Northbound from ovnkube-node-p9p6d...
2026-06-08T12:11:49.993322162Z INFO: Gathering OVN_Northbound from ovnkube-node-tvzqp...
2026-06-08T12:11:49.993565130Z INFO: Gathering OVN_Northbound from ovnkube-node-xr47f...
2026-06-08T12:11:49.993853194Z INFO: Gathering OVN_Southbound from ovnkube-node-p9p6d...
2026-06-08T12:11:49.994096536Z INFO: Gathering OVN_Southbound from ovnkube-node-tvzqp...
2026-06-08T12:11:49.994336766Z INFO: Gathering OVN_Southbound from ovnkube-node-xr47f...
2026-06-08T12:11:50.163736732Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:50.194068664Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-08T12:11:50.298696944Z Wrote inspect data to must-gather.
2026-06-08T12:11:50.317317195Z tar: Removing leading `/' from member names
2026-06-08T12:11:50.320665840Z tar: Removing leading `/' from member names
2026-06-08T12:11:50.349536840Z tar: Removing leading `/' from member names
2026-06-08T12:11:50.366338826Z tar: Removing leading `/' from member names
2026-06-08T12:11:50.371781114Z tar: Removing leading `/' from member names
2026-06-08T12:11:50.382754254Z tar: Removing leading `/' from member names
2026-06-08T12:11:50.429839293Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T12:11:50.491427695Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:50.495646725Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-08T12:11:50.513964936Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-08T12:11:50.530064109Z Wrote inspect data to must-gather.
2026-06-08T12:11:50.742131140Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:50.778809897Z Wrote inspect data to must-gather.
2026-06-08T12:11:50.852847231Z INFO: Waiting for network log collection to complete ...
2026-06-08T12:11:50.852900140Z INFO: Waiting for ovnk database copies to complete ...
2026-06-08T12:11:50.853400725Z INFO: Copying ovnk databases complete.
2026-06-08T12:11:50.855315578Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-06-08T12:11:50.858461772Z ovnk_database_store/
2026-06-08T12:11:50.858461772Z ovnk_database_store/ovnkube-node-p9p6d_nbdb
2026-06-08T12:11:50.870954520Z ovnk_database_store/ovnkube-node-tvzqp_sbdb
2026-06-08T12:11:50.890777275Z ovnk_database_store/ovnkube-node-p9p6d_sbdb
2026-06-08T12:11:50.915694460Z ovnk_database_store/ovnkube-node-xr47f_sbdb
2026-06-08T12:11:50.943734614Z ovnk_database_store/ovnkube-node-xr47f_nbdb
2026-06-08T12:11:50.955167361Z ovnk_database_store/ovnkube-node-tvzqp_nbdb
2026-06-08T12:11:50.962039571Z INFO: Getting status from alertmanager-main-0
2026-06-08T12:11:50.970135290Z INFO: Network log collection complete.
2026-06-08T12:11:50.982039090Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:50.985262389Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-08T12:11:51.014232462Z W0608 12:11:51.014183    1513 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-08T12:11:51.021282184Z Wrote inspect data to must-gather.
2026-06-08T12:11:51.121471789Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T12:11:51.172161273Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:51.292071530Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-08T12:11:51.294449997Z INFO: Fallback to identify the container image from release info
2026-06-08T12:11:51.313358603Z Gathering data for ns/openshift-network-console...
2026-06-08T12:11:51.365427973Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:51.368801331Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-08T12:11:51.572095278Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:51.720606601Z Gathering data for ns/openshift-console-operator...
2026-06-08T12:11:51.752687439Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:51.754937321Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-08T12:11:51.908236528Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:52.061145926Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-08T12:11:52.108312796Z Wrote inspect data to /must-gather/istio.
2026-06-08T12:11:52.122043066Z Gathering data for ns/openshift-console...
2026-06-08T12:11:52.179284046Z daemonset.apps/perf-node-gather-daemonset created
2026-06-08T12:11:52.322264102Z Waiting for performance profile collector pods to become ready: 1
2026-06-08T12:11:52.685296034Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-08T12:11:52.799003558Z Done executing Istio gather script
2026-06-08T12:11:53.135812989Z Gathering data for ns/openshift-dns-operator...
2026-06-08T12:11:53.389642349Z Gathering data for ns/openshift-dns...
2026-06-08T12:11:53.430058456Z Waiting for performance profile collector pods to become ready: 2
2026-06-08T12:11:53.746068570Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-08T12:11:53.919985115Z Gathering data for ns/openshift-image-registry...
2026-06-08T12:11:54.363119891Z Gathering data for ns/openshift-ingress-operator...
2026-06-08T12:11:54.537600724Z Waiting for performance profile collector pods to become ready: 3
2026-06-08T12:11:54.671083375Z Gathering data for ns/openshift-ingress...
2026-06-08T12:11:54.966940812Z Gathering data for ns/openshift-ingress-canary...
2026-06-08T12:11:55.316080440Z Gathering data for ns/openshift-insights...
2026-06-08T12:11:55.655263803Z Waiting for performance profile collector pods to become ready: 4
2026-06-08T12:11:56.768098626Z Waiting for performance profile collector pods to become ready: 5
2026-06-08T12:11:57.280127682Z Gathering data for ns/openshift-config...
2026-06-08T12:11:57.527667597Z Gathering data for ns/openshift-config-managed...
2026-06-08T12:11:57.883973186Z Waiting for performance profile collector pods to become ready: 6
2026-06-08T12:11:57.946976713Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-08T12:11:58.204340977Z Gathering data for ns/openshift-kube-apiserver...
2026-06-08T12:11:58.442867169Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-08T12:11:58.727152747Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-08T12:11:58.752413978Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-08T12:11:59.009415509Z Waiting for performance profile collector pods to become ready: 7
2026-06-08T12:11:59.042624685Z Gathering data for ns/openshift-kube-scheduler...
2026-06-08T12:11:59.342630644Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-08T12:11:59.621569407Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-08T12:11:59.968861743Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-08T12:12:00.120189162Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-08T12:12:00.360860694Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-08T12:12:00.365622041Z Collecting performance related data for node ip-10-0-135-253.ec2.internal
2026-06-08T12:12:00.374888781Z Collecting performance related data for node ip-10-0-132-76.ec2.internal
2026-06-08T12:12:00.387949820Z Collecting performance related data for node ip-10-0-137-95.ec2.internal
2026-06-08T12:12:00.934944033Z Gathering data for ns/openshift-multus...
2026-06-08T12:12:02.502592918Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-08T12:12:03.758265346Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-08T12:12:03.762479740Z Gathering data for ns/openshift-host-network...
2026-06-08T12:12:04.149580046Z Gathering data for ns/openshift-network-diagnostics...
2026-06-08T12:12:04.671833699Z Gathering data for ns/openshift-network-node-identity...
2026-06-08T12:12:05.060441699Z Gathering data for ns/openshift-network-operator...
2026-06-08T12:12:05.440774413Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-08T12:12:05.699031175Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-08T12:12:06.175348328Z Gathering data for ns/openshift-apiserver-operator...
2026-06-08T12:12:06.449013822Z Gathering data for ns/openshift-apiserver...
2026-06-08T12:12:06.803991566Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-08T12:12:07.045869023Z Gathering data for ns/openshift-controller-manager...
2026-06-08T12:12:07.307525811Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-08T12:12:07.976467147Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-08T12:12:08.152687857Z Collecting kubelet logs for node ip-10-0-135-253.ec2.internal
2026-06-08T12:12:08.155035021Z Collecting kubelet logs for node ip-10-0-132-76.ec2.internal
2026-06-08T12:12:08.157155470Z Collecting kubelet logs for node ip-10-0-137-95.ec2.internal
2026-06-08T12:12:08.352398472Z Gathering data for ns/openshift-service-ca-operator...
2026-06-08T12:12:08.520814956Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-b2hrr namespace
2026-06-08T12:12:08.529378940Z INFO: Node performance data collection complete.
2026-06-08T12:12:08.689974013Z Gathering data for ns/openshift-service-ca...
2026-06-08T12:12:08.764473016Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-08T12:12:09.065686083Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-08T12:12:10.013268444Z Wrote inspect data to must-gather.
2026-06-08T12:12:10.013305373Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T12:12:10.013305373Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-08T12:12:10.130675010Z error: the server doesn't have a resource type "clusters"
2026-06-08T12:12:10.259366568Z Caches written to disk