2026-06-15T06:40:14.938773594Z [disk usage checker] Started
2026-06-15T06:40:14.942116349Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-15T06:40:15.149116530Z Gathering data for ns/openshift-cluster-version...
2026-06-15T06:40:15.288445010Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-15T06:40:15.383429734Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-15T06:40:15.470660653Z Gathering data for ns/default...
2026-06-15T06:40:15.861435034Z Gathering data for ns/openshift...
2026-06-15T06:40:16.274410469Z Gathering data for ns/kube-system...
2026-06-15T06:40:16.994505440Z Gathering data for ns/openshift-etcd...
2026-06-15T06:40:17.347624795Z Wrote inspect data to must-gather.
2026-06-15T06:40:17.347657225Z error: inspection completed with the errors occurred while gathering data:
2026-06-15T06:40:17.347657225Z     namespaces "assisted-installer" not found
2026-06-15T06:40:17.786865880Z Waiting on subprocesses to finish execution.
2026-06-15T06:40:17.794923394Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-15T06:40:17.798142066Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-15T06:40:17.800211551Z INFO: Gathering HAProxy config files
2026-06-15T06:40:17.811094508Z INFO: Collecting host service logs for crio
2026-06-15T06:40:17.811431707Z INFO: Collecting host service logs for kubelet
2026-06-15T06:40:17.811755272Z INFO: Collecting host service logs for rpm-ostreed
2026-06-15T06:40:17.812051195Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-15T06:40:17.812384550Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-15T06:40:17.812722067Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-15T06:40:17.813064916Z INFO: Collecting host service logs for NetworkManager
2026-06-15T06:40:17.813440383Z INFO: Collecting host service logs for openvswitch
2026-06-15T06:40:17.813765550Z INFO: Collecting host service logs for ovs-configuration
2026-06-15T06:40:17.814117940Z INFO: Collecting host service logs for ovsdb-server
2026-06-15T06:40:17.814465725Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-15T06:40:17.814864481Z INFO: Waiting for worker host service log collection to complete ...
2026-06-15T06:40:17.815858227Z Executing Istio gather script
2026-06-15T06:40:17.817154693Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-15T06:40:17.955969274Z INFO: Waiting for node performance related collection to complete ...
2026-06-15T06:40:18.434269623Z INFO: Collecting Insights Archives from insights-operator-5bbd86d6bd-rf87f
2026-06-15T06:40:18.434322877Z insights-runtime-extractor-dvd67
2026-06-15T06:40:18.434345961Z insights-runtime-extractor-l4fhk
2026-06-15T06:40:18.434366359Z insights-runtime-extractor-sg2hl
2026-06-15T06:40:18.797936338Z No resources found in openshift-etcd namespace.
2026-06-15T06:40:18.924367993Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-15T06:40:19.069005648Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-15T06:40:19.069005648Z See 'oc rsync -h' for help and examples
2026-06-15T06:40:19.115619681Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-15T06:40:19.168104235Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-15T06:40:19.180240842Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-15T06:40:19.180240842Z INFO: on-disk MachineConfig config collection complete.
2026-06-15T06:40:19.180373591Z INFO: "metallb-operator" not detected. Skipping.
2026-06-15T06:40:19.197653885Z Wrote inspect data to must-gather.
2026-06-15T06:40:19.208690941Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-15T06:40:19.407872144Z No resources found
2026-06-15T06:40:19.446036816Z INFO: Worker host service log collection to complete.
2026-06-15T06:40:19.451600551Z ERROR: No running kube-apiserver pods found
2026-06-15T06:40:19.470416340Z error: the server doesn't have a resource type "performanceprofile"
2026-06-15T06:40:19.519349449Z Wrote inspect data to must-gather.
2026-06-15T06:40:19.545575709Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-15T06:40:19.893076948Z Wrote inspect data to must-gather.
2026-06-15T06:40:19.932900766Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-15T06:40:19.982913519Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-15T06:40:19.990719907Z INFO: 'previous-logs' folder not found on ip-10-0-129-86.ec2.internal, skipping...
2026-06-15T06:40:20.067098924Z No resources found
2026-06-15T06:40:20.168309740Z Wrote inspect data to must-gather.
2026-06-15T06:40:20.252825047Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-15T06:40:20.258840557Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-15T06:40:20.439102064Z Wrote inspect data to must-gather.
2026-06-15T06:40:20.481668384Z tar: Removing leading `/' from member names
2026-06-15T06:40:20.493550334Z INFO: HAProxy config collection complete.
2026-06-15T06:40:20.530461389Z Wrote inspect data to must-gather.
2026-06-15T06:40:20.605008633Z Wrote inspect data to must-gather.
2026-06-15T06:40:20.628127662Z INFO: 'previous-logs' folder not found on ip-10-0-132-4.ec2.internal, skipping...
2026-06-15T06:40:20.676645919Z Wrote inspect data to must-gather.
2026-06-15T06:40:20.731345099Z error: the server doesn't have a resource type "machineconfigs"
2026-06-15T06:40:20.812693439Z Gathering data for ns/openshift-monitoring...
2026-06-15T06:40:20.892441678Z INFO: Getting rules from prometheus-k8s-0
2026-06-15T06:40:20.954180699Z INFO: 'previous-logs' folder not found on ip-10-0-133-111.ec2.internal, skipping...
2026-06-15T06:40:20.954293867Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-15T06:40:20.954316811Z INFO: Machine Config Daemon termination log collection complete.
2026-06-15T06:40:20.960920546Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-15T06:40:20.993789555Z Wrote inspect data to must-gather.
2026-06-15T06:40:21.195399931Z Wrote inspect data to must-gather.
2026-06-15T06:40:21.271150908Z Wrote inspect data to must-gather.
2026-06-15T06:40:21.274706797Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-15T06:40:21.366339269Z INFO: Getting status/config from prometheus-k8s-0
2026-06-15T06:40:21.379310454Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-15T06:40:21.600312813Z INFO: INTERCONNECT MODE
2026-06-15T06:40:21.600345081Z INFO: Gathering ovn-kubernetes DBs
2026-06-15T06:40:21.611687745Z Wrote inspect data to must-gather.
2026-06-15T06:40:21.776392509Z INFO: Gathering OVN_Northbound from ovnkube-node-gb7k9...
2026-06-15T06:40:21.776675879Z INFO: Gathering OVN_Northbound from ovnkube-node-h6dj8...
2026-06-15T06:40:21.776916320Z INFO: Gathering OVN_Northbound from ovnkube-node-rm8q8...
2026-06-15T06:40:21.777279791Z INFO: Gathering OVN_Southbound from ovnkube-node-gb7k9...
2026-06-15T06:40:21.777524587Z INFO: Gathering OVN_Southbound from ovnkube-node-h6dj8...
2026-06-15T06:40:21.777843500Z INFO: Gathering OVN_Southbound from ovnkube-node-rm8q8...
2026-06-15T06:40:21.858060278Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-15T06:40:21.891137769Z Wrote inspect data to must-gather.
2026-06-15T06:40:22.128571267Z tar: Removing leading `/' from member names
2026-06-15T06:40:22.133688252Z tar: Removing leading `/' from member names
2026-06-15T06:40:22.133748118Z tar: Removing leading `/' from member names
2026-06-15T06:40:22.135926866Z tar: Removing leading `/' from member names
2026-06-15T06:40:22.144119555Z tar: Removing leading `/' from member names
2026-06-15T06:40:22.204918376Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-15T06:40:22.205601885Z Wrote inspect data to must-gather.
2026-06-15T06:40:22.321448933Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-15T06:40:22.483515812Z Wrote inspect data to must-gather.
2026-06-15T06:40:22.486035827Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-15T06:40:22.570632563Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:22.575660056Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-15T06:40:22.720633295Z INFO: Waiting for network log collection to complete ...
2026-06-15T06:40:22.720670232Z INFO: Waiting for ovnk database copies to complete ...
2026-06-15T06:40:22.721116762Z INFO: Copying ovnk databases complete.
2026-06-15T06:40:22.723358283Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-06-15T06:40:22.726050989Z ovnk_database_store/
2026-06-15T06:40:22.726098944Z ovnk_database_store/ovnkube-node-h6dj8_sbdb
2026-06-15T06:40:22.728043781Z Wrote inspect data to must-gather.
2026-06-15T06:40:22.749326282Z ovnk_database_store/ovnkube-node-h6dj8_nbdb
2026-06-15T06:40:22.760420239Z ovnk_database_store/ovnkube-node-rm8q8_nbdb
2026-06-15T06:40:22.767303889Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-15T06:40:22.772001199Z ovnk_database_store/ovnkube-node-rm8q8_sbdb
2026-06-15T06:40:22.796915740Z ovnk_database_store/ovnkube-node-gb7k9_nbdb
2026-06-15T06:40:22.808288397Z ovnk_database_store/ovnkube-node-gb7k9_sbdb
2026-06-15T06:40:22.811502873Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:22.814574051Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-15T06:40:22.836308698Z INFO: Network log collection complete.
2026-06-15T06:40:22.980426312Z Wrote inspect data to must-gather.
2026-06-15T06:40:23.041472921Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:23.196812956Z INFO: Getting status from alertmanager-main-0
2026-06-15T06:40:23.206554786Z W0615 06:40:23.206498    1511 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-15T06:40:23.213052471Z Wrote inspect data to must-gather.
2026-06-15T06:40:23.227447262Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-15T06:40:23.275718715Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:23.279103201Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-15T06:40:23.426316177Z Gathering data for ns/openshift-network-console...
2026-06-15T06:40:23.520730943Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-15T06:40:23.523499274Z INFO: Fallback to identify the container image from release info
2026-06-15T06:40:23.527602369Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:23.742041594Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:23.744835428Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-15T06:40:23.880846733Z Gathering data for ns/openshift-console-operator...
2026-06-15T06:40:23.987518068Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:24.189876241Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:24.193756409Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io
2026-06-15T06:40:24.200178494Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d66a6b2ec52dede5d83ce04f6abfc519df1fd7dc36248741e03ecb9bf8d1762
2026-06-15T06:40:24.313485484Z Gathering data for ns/openshift-console...
2026-06-15T06:40:24.354010872Z daemonset.apps/perf-node-gather-daemonset created
2026-06-15T06:40:24.393231425Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:24.524403774Z Waiting for performance profile collector pods to become ready: 1
2026-06-15T06:40:24.617305283Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:24.621017933Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io
2026-06-15T06:40:24.830780470Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:24.936762128Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-15T06:40:24.992150586Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-15T06:40:25.060857867Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:25.063419808Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io
2026-06-15T06:40:25.226226159Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:25.439819839Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:25.442145057Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-06-15T06:40:25.489295536Z Gathering data for ns/openshift-dns-operator...
2026-06-15T06:40:25.637432711Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:25.663396231Z Waiting for performance profile collector pods to become ready: 2
2026-06-15T06:40:25.847164225Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:25.848095662Z Gathering data for ns/openshift-dns...
2026-06-15T06:40:25.849720860Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-06-15T06:40:26.005362155Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:26.205348766Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:26.208292686Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-15T06:40:26.382963900Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:26.508128868Z Gathering data for ns/openshift-image-registry...
2026-06-15T06:40:26.579643021Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:40:26.796733684Z Waiting for performance profile collector pods to become ready: 3
2026-06-15T06:40:27.034293931Z Gathering data for ns/openshift-ingress-operator...
2026-06-15T06:40:27.315048539Z Done executing Istio gather script
2026-06-15T06:40:27.409535843Z Gathering data for ns/openshift-ingress...
2026-06-15T06:40:27.803306639Z Gathering data for ns/openshift-ingress-canary...
2026-06-15T06:40:27.920421198Z Waiting for performance profile collector pods to become ready: 4
2026-06-15T06:40:28.213995426Z Gathering data for ns/openshift-insights...
2026-06-15T06:40:29.041346722Z Waiting for performance profile collector pods to become ready: 5
2026-06-15T06:40:29.997134675Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-15T06:40:30.169155223Z Waiting for performance profile collector pods to become ready: 6
2026-06-15T06:40:30.342321728Z Gathering data for ns/kserve...
2026-06-15T06:40:31.314820423Z Waiting for performance profile collector pods to become ready: 7
2026-06-15T06:40:31.930833420Z Gathering data for ns/openshift-config...
2026-06-15T06:40:32.245108199Z Gathering data for ns/openshift-config-managed...
2026-06-15T06:40:32.431895685Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-15T06:40:32.696136920Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-15T06:40:32.712092376Z Collecting performance related data for node ip-10-0-133-111.ec2.internal
2026-06-15T06:40:32.720871022Z Collecting performance related data for node ip-10-0-132-4.ec2.internal
2026-06-15T06:40:32.732349277Z Collecting performance related data for node ip-10-0-129-86.ec2.internal
2026-06-15T06:40:33.025198953Z Gathering data for ns/openshift-kube-apiserver...
2026-06-15T06:40:33.368280709Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-15T06:40:33.837973324Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-15T06:40:34.190224323Z Gathering data for ns/openshift-kube-scheduler...
2026-06-15T06:40:34.517607868Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-15T06:40:34.815953032Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-15T06:40:35.002544484Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-15T06:40:35.179622214Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-15T06:40:35.545312301Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-15T06:40:36.104333282Z Gathering data for ns/openshift-multus...
2026-06-15T06:40:37.565570577Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-15T06:40:38.859815645Z Gathering data for ns/openshift-host-network...
2026-06-15T06:40:39.185830076Z Gathering data for ns/openshift-network-diagnostics...
2026-06-15T06:40:39.814164152Z Gathering data for ns/openshift-network-node-identity...
2026-06-15T06:40:40.011678898Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-15T06:40:40.281240760Z Gathering data for ns/openshift-network-operator...
2026-06-15T06:40:40.682964000Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-15T06:40:41.009087271Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-15T06:40:41.040395311Z Collecting kubelet logs for node ip-10-0-133-111.ec2.internal
2026-06-15T06:40:41.042316371Z Collecting kubelet logs for node ip-10-0-132-4.ec2.internal
2026-06-15T06:40:41.044122917Z Collecting kubelet logs for node ip-10-0-129-86.ec2.internal
2026-06-15T06:40:41.419374428Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-j8rxx namespace
2026-06-15T06:40:41.424968103Z INFO: Node performance data collection complete.
2026-06-15T06:40:41.573070481Z Gathering data for ns/openshift-apiserver-operator...
2026-06-15T06:40:41.890612896Z Gathering data for ns/openshift-apiserver...
2026-06-15T06:40:42.289877810Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-15T06:40:42.596591912Z Gathering data for ns/openshift-controller-manager...
2026-06-15T06:40:42.919341570Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-15T06:40:43.564408015Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-15T06:40:43.899788944Z Gathering data for ns/openshift-service-ca-operator...
2026-06-15T06:40:44.215122074Z Gathering data for ns/openshift-service-ca...
2026-06-15T06:40:44.584521962Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-15T06:40:45.017964439Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-06-15T06:40:45.521291154Z Wrote inspect data to must-gather.
2026-06-15T06:40:45.521343885Z error: inspection completed with the errors occurred while gathering data:
2026-06-15T06:40:45.521343885Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-15T06:40:45.659133345Z error: the server doesn't have a resource type "clusters"
2026-06-15T06:40:45.789302540Z Caches written to disk