2026-06-09T13:39:12.353624916Z [disk usage checker] Started
2026-06-09T13:39:12.357352028Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T13:39:12.554491457Z Gathering data for ns/openshift-cluster-version...
2026-06-09T13:39:12.678207289Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T13:39:12.759218438Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T13:39:12.833827375Z Gathering data for ns/default...
2026-06-09T13:39:13.119548508Z Gathering data for ns/openshift...
2026-06-09T13:39:13.527812508Z Gathering data for ns/kube-system...
2026-06-09T13:39:14.179359757Z Gathering data for ns/openshift-etcd...
2026-06-09T13:39:14.470954558Z Wrote inspect data to must-gather.
2026-06-09T13:39:14.470992539Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T13:39:14.470992539Z     namespaces "assisted-installer" not found
2026-06-09T13:39:14.910372019Z Waiting on subprocesses to finish execution.
2026-06-09T13:39:14.915835786Z INFO: Collecting host service logs for crio
2026-06-09T13:39:14.916219647Z INFO: Collecting host service logs for kubelet
2026-06-09T13:39:14.916526644Z INFO: Collecting host service logs for rpm-ostreed
2026-06-09T13:39:14.916814055Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-09T13:39:14.917521801Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-09T13:39:14.917885978Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-09T13:39:14.918227491Z INFO: Collecting host service logs for NetworkManager
2026-06-09T13:39:14.918566683Z INFO: Collecting host service logs for openvswitch
2026-06-09T13:39:14.918898503Z INFO: Collecting host service logs for ovs-configuration
2026-06-09T13:39:14.919246569Z Executing Istio gather script
2026-06-09T13:39:14.919246569Z INFO: Collecting host service logs for ovsdb-server
2026-06-09T13:39:14.919600458Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-09T13:39:14.919987709Z INFO: Waiting for worker host service log collection to complete ...
2026-06-09T13:39:14.949846923Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-09T13:39:14.952576699Z INFO: Gathering HAProxy config files
2026-06-09T13:39:14.963923149Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-09T13:39:14.972734500Z INFO: Waiting for node performance related collection to complete ...
2026-06-09T13:39:15.105921951Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-09T13:39:15.534771301Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-09T13:39:15.621109876Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-09T13:39:15.622226925Z INFO: on-disk MachineConfig config collection complete.
2026-06-09T13:39:15.800929306Z INFO: Collecting Insights Archives from insights-operator-896549976-986rs
2026-06-09T13:39:15.800984865Z insights-runtime-extractor-bhlps
2026-06-09T13:39:15.801006157Z insights-runtime-extractor-n46nj
2026-06-09T13:39:15.801081968Z insights-runtime-extractor-trqkg
2026-06-09T13:39:15.977921096Z No resources found
2026-06-09T13:39:15.986489491Z INFO: "metallb-operator" not detected. Skipping.
2026-06-09T13:39:16.073910328Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-09T13:39:16.083173946Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-09T13:39:16.120828442Z Wrote inspect data to must-gather.
2026-06-09T13:39:16.140067489Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-09T13:39:16.171302823Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-09T13:39:16.273739946Z No resources found in openshift-etcd namespace.
2026-06-09T13:39:16.455540364Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-09T13:39:16.565860292Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T13:39:16.766194915Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-09T13:39:16.936841532Z Wrote inspect data to must-gather.
2026-06-09T13:39:16.937152015Z INFO: Worker host service log collection to complete.
2026-06-09T13:39:16.973981919Z error: the server doesn't have a resource type "performanceprofile"
2026-06-09T13:39:17.008527927Z Wrote inspect data to must-gather.
2026-06-09T13:39:17.083690999Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-09T13:39:17.083690999Z See 'oc rsync -h' for help and examples
2026-06-09T13:39:17.088604237Z No resources found
2026-06-09T13:39:17.106509788Z Wrote inspect data to must-gather.
2026-06-09T13:39:17.137556498Z INFO: 'previous-logs' folder not found on ip-10-0-128-194.ec2.internal, skipping...
2026-06-09T13:39:17.210863409Z Wrote inspect data to must-gather.
2026-06-09T13:39:17.218749711Z ERROR: No running kube-apiserver pods found
2026-06-09T13:39:17.239760940Z Wrote inspect data to must-gather.
2026-06-09T13:39:17.298070082Z tar: Removing leading `/' from member names
2026-06-09T13:39:17.305345356Z INFO: HAProxy config collection complete.
2026-06-09T13:39:17.385345377Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T13:39:17.441502369Z Wrote inspect data to must-gather.
2026-06-09T13:39:17.505410514Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-09T13:39:17.520975651Z INFO: 'previous-logs' folder not found on ip-10-0-135-182.ec2.internal, skipping...
2026-06-09T13:39:17.585065470Z Wrote inspect data to must-gather.
2026-06-09T13:39:17.629613157Z error: the server doesn't have a resource type "machineconfigs"
2026-06-09T13:39:17.656916446Z Gathering data for ns/openshift-monitoring...
2026-06-09T13:39:17.718237982Z INFO: Getting rules from prometheus-k8s-0
2026-06-09T13:39:17.822498467Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-09T13:39:17.839221861Z INFO: 'previous-logs' folder not found on ip-10-0-137-50.ec2.internal, skipping...
2026-06-09T13:39:17.839261968Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-09T13:39:17.839297149Z INFO: Machine Config Daemon termination log collection complete.
2026-06-09T13:39:17.872017078Z Wrote inspect data to must-gather.
2026-06-09T13:39:18.054042151Z Wrote inspect data to must-gather.
2026-06-09T13:39:18.121708070Z Wrote inspect data to must-gather.
2026-06-09T13:39:18.124321947Z INFO: Getting status/config from prometheus-k8s-0
2026-06-09T13:39:18.217724035Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-09T13:39:18.278305662Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-09T13:39:18.411094587Z INFO: INTERCONNECT MODE
2026-06-09T13:39:18.411183059Z INFO: Gathering ovn-kubernetes DBs
2026-06-09T13:39:18.446959764Z Wrote inspect data to must-gather.
2026-06-09T13:39:18.459444501Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:18.520197478Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-09T13:39:18.580055855Z INFO: Gathering OVN_Northbound from ovnkube-node-47mdr...
2026-06-09T13:39:18.580499351Z INFO: Gathering OVN_Northbound from ovnkube-node-ksk29...
2026-06-09T13:39:18.580765081Z INFO: Gathering OVN_Northbound from ovnkube-node-x7t4h...
2026-06-09T13:39:18.581093326Z INFO: Gathering OVN_Southbound from ovnkube-node-47mdr...
2026-06-09T13:39:18.581363074Z INFO: Gathering OVN_Southbound from ovnkube-node-ksk29...
2026-06-09T13:39:18.581653202Z INFO: Gathering OVN_Southbound from ovnkube-node-x7t4h...
2026-06-09T13:39:18.729717645Z Wrote inspect data to must-gather.
2026-06-09T13:39:18.765798008Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:18.769206444Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-09T13:39:18.857278082Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-09T13:39:18.900341118Z tar: Removing leading `/' from member names
2026-06-09T13:39:18.919248651Z tar: Removing leading `/' from member names
2026-06-09T13:39:18.942623351Z tar: Removing leading `/' from member names
2026-06-09T13:39:18.945730424Z tar: Removing leading `/' from member names
2026-06-09T13:39:18.947272171Z tar: Removing leading `/' from member names
2026-06-09T13:39:18.951421895Z tar: Removing leading `/' from member names
2026-06-09T13:39:19.067203054Z Wrote inspect data to must-gather.
2026-06-09T13:39:19.073272563Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:19.137082974Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-09T13:39:19.144052595Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T13:39:19.306922184Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:19.309627543Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-09T13:39:19.324850535Z Wrote inspect data to must-gather.
2026-06-09T13:39:19.418208107Z INFO: Waiting for network log collection to complete ...
2026-06-09T13:39:19.418320635Z INFO: Waiting for ovnk database copies to complete ...
2026-06-09T13:39:19.418835626Z INFO: Copying ovnk databases complete.
2026-06-09T13:39:19.420824108Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-06-09T13:39:19.420918626Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-09T13:39:19.423661431Z ovnk_database_store/
2026-06-09T13:39:19.423661431Z ovnk_database_store/ovnkube-node-x7t4h_nbdb
2026-06-09T13:39:19.432653233Z ovnk_database_store/ovnkube-node-ksk29_sbdb
2026-06-09T13:39:19.450773750Z ovnk_database_store/ovnkube-node-x7t4h_sbdb
2026-06-09T13:39:19.466629543Z ovnk_database_store/ovnkube-node-47mdr_sbdb
2026-06-09T13:39:19.485105460Z ovnk_database_store/ovnkube-node-ksk29_nbdb
2026-06-09T13:39:19.492610258Z ovnk_database_store/ovnkube-node-47mdr_nbdb
2026-06-09T13:39:19.504584849Z INFO: Network log collection complete.
2026-06-09T13:39:19.518052745Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:19.560055803Z Wrote inspect data to must-gather.
2026-06-09T13:39:19.727238035Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:19.730439856Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-09T13:39:19.777999539Z W0609 13:39:19.777955    1550 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-09T13:39:19.784864884Z Wrote inspect data to must-gather.
2026-06-09T13:39:19.801554696Z INFO: Getting status from alertmanager-main-0
2026-06-09T13:39:19.806159223Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T13:39:19.980118289Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:20.003164639Z Gathering data for ns/openshift-network-console...
2026-06-09T13:39:20.099124154Z Wrote inspect data to must-gather.
2026-06-09T13:39:20.186081393Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:20.189875830Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-09T13:39:20.365579250Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-09T13:39:20.368049243Z INFO: Fallback to identify the container image from release info
2026-06-09T13:39:20.381046391Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:20.402808780Z Gathering data for ns/openshift-console-operator...
2026-06-09T13:39:20.574560465Z Wrote inspect data to /must-gather/istio.
2026-06-09T13:39:20.777744436Z Gathering data for ns/openshift-console...
2026-06-09T13:39:20.927643040Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-09T13:39:21.046759900Z daemonset.apps/perf-node-gather-daemonset created
2026-06-09T13:39:21.184230670Z Waiting for performance profile collector pods to become ready: 1
2026-06-09T13:39:21.267104545Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-09T13:39:21.277552283Z Done executing Istio gather script
2026-06-09T13:39:21.715212020Z Gathering data for ns/openshift-dns-operator...
2026-06-09T13:39:21.951957317Z Gathering data for ns/openshift-dns...
2026-06-09T13:39:22.299460431Z Waiting for performance profile collector pods to become ready: 2
2026-06-09T13:39:22.391297714Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T13:39:22.507220928Z Gathering data for ns/openshift-image-registry...
2026-06-09T13:39:22.915623556Z Gathering data for ns/openshift-ingress-operator...
2026-06-09T13:39:23.192170080Z Gathering data for ns/openshift-ingress...
2026-06-09T13:39:23.413862716Z Waiting for performance profile collector pods to become ready: 3
2026-06-09T13:39:23.490332316Z Gathering data for ns/openshift-ingress-canary...
2026-06-09T13:39:23.847957894Z Gathering data for ns/openshift-insights...
2026-06-09T13:39:24.536184374Z Waiting for performance profile collector pods to become ready: 4
2026-06-09T13:39:25.578660505Z Gathering data for ns/openshift-config...
2026-06-09T13:39:25.659137955Z Waiting for performance profile collector pods to become ready: 5
2026-06-09T13:39:25.817660863Z Gathering data for ns/openshift-config-managed...
2026-06-09T13:39:26.175763345Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-09T13:39:26.406508731Z Gathering data for ns/openshift-kube-apiserver...
2026-06-09T13:39:26.632641572Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-09T13:39:26.775152561Z Waiting for performance profile collector pods to become ready: 6
2026-06-09T13:39:26.889451882Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-09T13:39:27.145697552Z Gathering data for ns/openshift-kube-scheduler...
2026-06-09T13:39:27.367665305Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-09T13:39:27.397978382Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T13:39:27.607984170Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-09T13:39:27.900519317Z Waiting for performance profile collector pods to become ready: 7
2026-06-09T13:39:27.902311141Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-09T13:39:28.195139068Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-09T13:39:28.670816751Z Gathering data for ns/openshift-multus...
2026-06-09T13:39:29.000314802Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-09T13:39:29.217648835Z Collecting performance related data for node ip-10-0-137-50.ec2.internal
2026-06-09T13:39:29.224240658Z Collecting performance related data for node ip-10-0-135-182.ec2.internal
2026-06-09T13:39:29.232386098Z Collecting performance related data for node ip-10-0-128-194.ec2.internal
2026-06-09T13:39:30.081584818Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-09T13:39:31.226501148Z Gathering data for ns/openshift-host-network...
2026-06-09T13:39:31.518566377Z Gathering data for ns/openshift-network-diagnostics...
2026-06-09T13:39:32.038351865Z Gathering data for ns/openshift-network-node-identity...
2026-06-09T13:39:32.403814716Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T13:39:32.409074220Z Gathering data for ns/openshift-network-operator...
2026-06-09T13:39:32.749333977Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-09T13:39:32.978976596Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-09T13:39:33.430992548Z Gathering data for ns/openshift-apiserver-operator...
2026-06-09T13:39:33.711704990Z Gathering data for ns/openshift-apiserver...
2026-06-09T13:39:34.100488437Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-09T13:39:34.339197423Z Gathering data for ns/openshift-controller-manager...
2026-06-09T13:39:34.585900338Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-09T13:39:35.208970346Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-09T13:39:35.509012437Z Gathering data for ns/openshift-service-ca-operator...
2026-06-09T13:39:35.777548252Z Gathering data for ns/openshift-service-ca...
2026-06-09T13:39:36.095286061Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-09T13:39:36.922388340Z Wrote inspect data to must-gather.
2026-06-09T13:39:36.922432185Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T13:39:36.922432185Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-09T13:39:37.327068239Z Collecting kubelet logs for node ip-10-0-137-50.ec2.internal
2026-06-09T13:39:37.328686203Z Collecting kubelet logs for node ip-10-0-135-182.ec2.internal
2026-06-09T13:39:37.330426521Z Collecting kubelet logs for node ip-10-0-128-194.ec2.internal
2026-06-09T13:39:37.411244179Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T13:39:37.627218806Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-c8jjk namespace
2026-06-09T13:39:37.631682016Z INFO: Node performance data collection complete.
2026-06-09T13:39:37.755096366Z error: the server doesn't have a resource type "clusters"
2026-06-09T13:39:37.908295764Z Caches written to disk