2026-09-18T20:07:21.454544431Z [disk usage checker] Started
2026-09-18T20:07:21.457919431Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-18T20:07:21.659653330Z Gathering data for ns/openshift-cluster-version...
2026-09-18T20:07:21.793895343Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T20:07:21.879019436Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T20:07:21.961254645Z Gathering data for ns/default...
2026-09-18T20:07:22.267065218Z Gathering data for ns/openshift...
2026-09-18T20:07:22.663461652Z Gathering data for ns/kube-system...
2026-09-18T20:07:23.341916207Z Gathering data for ns/openshift-etcd...
2026-09-18T20:07:23.666301914Z Wrote inspect data to must-gather.
2026-09-18T20:07:23.666330110Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T20:07:23.666330110Z     namespaces "assisted-installer" not found
2026-09-18T20:07:24.177462156Z Waiting on subprocesses to finish execution.
2026-09-18T20:07:24.181933740Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-18T20:07:24.184482405Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-18T20:07:24.192229096Z INFO: Gathering HAProxy config files
2026-09-18T20:07:24.203689436Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-18T20:07:24.206975105Z Executing Istio gather script
2026-09-18T20:07:24.212510536Z INFO: Collecting host service logs for crio
2026-09-18T20:07:24.212842449Z INFO: Collecting host service logs for kubelet
2026-09-18T20:07:24.213207673Z INFO: Collecting host service logs for rpm-ostreed
2026-09-18T20:07:24.213567871Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-18T20:07:24.213908370Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-18T20:07:24.214280152Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-18T20:07:24.214638558Z INFO: Collecting host service logs for NetworkManager
2026-09-18T20:07:24.214976850Z INFO: Collecting host service logs for openvswitch
2026-09-18T20:07:24.220829354Z INFO: Collecting host service logs for ovs-configuration
2026-09-18T20:07:24.221231312Z INFO: Collecting host service logs for ovsdb-server
2026-09-18T20:07:24.221569312Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-18T20:07:24.222932211Z INFO: Waiting for worker host service log collection to complete ...
2026-09-18T20:07:24.652194117Z INFO: Waiting for node performance related collection to complete ...
2026-09-18T20:07:24.738447730Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-18T20:07:24.738516877Z INFO: on-disk MachineConfig config collection complete.
2026-09-18T20:07:24.847054022Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-18T20:07:24.989518711Z Wrote inspect data to must-gather.
2026-09-18T20:07:24.989763590Z INFO: "metallb-operator" not detected. Skipping.
2026-09-18T20:07:25.237001499Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-4vsvd
2026-09-18T20:07:25.237052678Z insights-runtime-extractor-6x52b
2026-09-18T20:07:25.237070593Z insights-runtime-extractor-lth96
2026-09-18T20:07:25.237088503Z insights-runtime-extractor-m5gtc
2026-09-18T20:07:25.389104192Z No resources found
2026-09-18T20:07:25.441436773Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-18T20:07:25.455810660Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-18T20:07:25.481255157Z ERROR: No running kube-apiserver pods found
2026-09-18T20:07:25.563488847Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-18T20:07:25.593886019Z Wrote inspect data to must-gather.
2026-09-18T20:07:25.617488856Z error: the server doesn't have a resource type "performanceprofile"
2026-09-18T20:07:25.618284610Z No resources found in openshift-etcd namespace.
2026-09-18T20:07:25.710171533Z INFO: Worker host service log collection to complete.
2026-09-18T20:07:25.887850654Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-18T20:07:25.887850654Z See 'oc rsync -h' for help and examples
2026-09-18T20:07:26.040367590Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T20:07:26.349603031Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-18T20:07:26.350446453Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-18T20:07:26.377619025Z Wrote inspect data to must-gather.
2026-09-18T20:07:26.498226173Z Gathering data for ns/openshift-monitoring...
2026-09-18T20:07:26.527298598Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-18T20:07:26.529454388Z No resources found
2026-09-18T20:07:26.654318275Z Wrote inspect data to must-gather.
2026-09-18T20:07:26.670768797Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-18T20:07:26.703304214Z Wrote inspect data to must-gather.
2026-09-18T20:07:26.740719483Z Wrote inspect data to must-gather.
2026-09-18T20:07:26.753866782Z Wrote inspect data to must-gather.
2026-09-18T20:07:26.815160103Z tar: Removing leading `/' from member names
2026-09-18T20:07:26.818729015Z INFO: 'previous-logs' folder not found on ip-10-0-130-199.ec2.internal, skipping...
2026-09-18T20:07:26.819425954Z INFO: HAProxy config collection complete.
2026-09-18T20:07:26.932026196Z Wrote inspect data to must-gather.
2026-09-18T20:07:26.948059441Z error: the server doesn't have a resource type "machineconfigs"
2026-09-18T20:07:27.147229434Z INFO: 'previous-logs' folder not found on ip-10-0-141-190.ec2.internal, skipping...
2026-09-18T20:07:27.151147800Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-18T20:07:27.198207057Z INFO: Getting rules from prometheus-k8s-0
2026-09-18T20:07:27.251683219Z Wrote inspect data to must-gather.
2026-09-18T20:07:27.417974737Z Wrote inspect data to must-gather.
2026-09-18T20:07:27.461548540Z INFO: 'previous-logs' folder not found on ip-10-0-143-198.ec2.internal, skipping...
2026-09-18T20:07:27.461646986Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-18T20:07:27.461695986Z INFO: Machine Config Daemon termination log collection complete.
2026-09-18T20:07:27.538923363Z Wrote inspect data to must-gather.
2026-09-18T20:07:27.621364388Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-18T20:07:27.656465818Z INFO: Getting status/config from prometheus-k8s-0
2026-09-18T20:07:27.718579075Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-18T20:07:27.869931885Z INFO: INTERCONNECT MODE
2026-09-18T20:07:27.869984358Z INFO: Gathering ovn-kubernetes DBs
2026-09-18T20:07:27.870310207Z Wrote inspect data to must-gather.
2026-09-18T20:07:28.080373798Z INFO: Gathering OVN_Northbound from ovnkube-node-2lf2j...
2026-09-18T20:07:28.080577899Z INFO: Gathering OVN_Northbound from ovnkube-node-b7nd9...
2026-09-18T20:07:28.080765034Z INFO: Gathering OVN_Northbound from ovnkube-node-dt7qz...
2026-09-18T20:07:28.081039243Z INFO: Gathering OVN_Southbound from ovnkube-node-2lf2j...
2026-09-18T20:07:28.081300754Z INFO: Gathering OVN_Southbound from ovnkube-node-b7nd9...
2026-09-18T20:07:28.081596678Z INFO: Gathering OVN_Southbound from ovnkube-node-dt7qz...
2026-09-18T20:07:28.101189524Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T20:07:28.113355587Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-18T20:07:28.217775881Z Wrote inspect data to must-gather.
2026-09-18T20:07:28.350357494Z tar: Removing leading `/' from member names
2026-09-18T20:07:28.368418295Z tar: Removing leading `/' from member names
2026-09-18T20:07:28.407306239Z tar: Removing leading `/' from member names
2026-09-18T20:07:28.413575545Z tar: Removing leading `/' from member names
2026-09-18T20:07:28.427676193Z tar: Removing leading `/' from member names
2026-09-18T20:07:28.499060366Z tar: Removing leading `/' from member names
2026-09-18T20:07:28.549437730Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-18T20:07:28.556864910Z Wrote inspect data to must-gather.
2026-09-18T20:07:28.810573408Z W0918 20:07:28.810520    1380 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-18T20:07:28.818532439Z Wrote inspect data to must-gather.
2026-09-18T20:07:28.821231439Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-18T20:07:28.884550501Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-18T20:07:28.898306070Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T20:07:28.984023813Z INFO: Waiting for network log collection to complete ...
2026-09-18T20:07:28.984090662Z INFO: Waiting for ovnk database copies to complete ...
2026-09-18T20:07:28.984645753Z INFO: Copying ovnk databases complete.
2026-09-18T20:07:28.988060359Z 3.8M	must-gather/network_logs/ovnk_database_store
2026-09-18T20:07:28.990807333Z ovnk_database_store/
2026-09-18T20:07:28.990858921Z ovnk_database_store/ovnkube-node-2lf2j_sbdb
2026-09-18T20:07:29.019666958Z ovnk_database_store/ovnkube-node-b7nd9_nbdb
2026-09-18T20:07:29.028794754Z ovnk_database_store/ovnkube-node-b7nd9_sbdb
2026-09-18T20:07:29.054713834Z ovnk_database_store/ovnkube-node-dt7qz_sbdb
2026-09-18T20:07:29.077116882Z ovnk_database_store/ovnkube-node-2lf2j_nbdb
2026-09-18T20:07:29.086231110Z ovnk_database_store/ovnkube-node-dt7qz_nbdb
2026-09-18T20:07:29.098109135Z INFO: Network log collection complete.
2026-09-18T20:07:29.109912222Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:29.111100688Z Gathering data for ns/openshift-network-console...
2026-09-18T20:07:29.113908274Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-18T20:07:29.190548656Z Wrote inspect data to must-gather.
2026-09-18T20:07:29.331345162Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:29.336462499Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-18T20:07:29.535427032Z Wrote inspect data to must-gather.
2026-09-18T20:07:29.543467154Z INFO: Getting status from alertmanager-main-0
2026-09-18T20:07:29.582551371Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:29.611189512Z Gathering data for ns/openshift-console-operator...
2026-09-18T20:07:29.810488689Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:29.813422091Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-18T20:07:29.850266586Z Wrote inspect data to must-gather.
2026-09-18T20:07:30.027169858Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:30.074100992Z Gathering data for ns/openshift-console...
2026-09-18T20:07:30.125824332Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-18T20:07:30.129035814Z INFO: Fallback to identify the container image from release info
2026-09-18T20:07:30.249999842Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:30.252423974Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-18T20:07:30.486901155Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:30.615587510Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-18T20:07:30.688290487Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:30.691102389Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-18T20:07:30.812254296Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-18T20:07:30.881104335Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:30.965167711Z daemonset.apps/perf-node-gather-daemonset created
2026-09-18T20:07:31.106537679Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:31.109892239Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-18T20:07:31.121837330Z Waiting for performance profile collector pods to become ready: 1
2026-09-18T20:07:31.175154377Z Gathering data for ns/openshift-dns-operator...
2026-09-18T20:07:31.289039223Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:31.513649082Z Gathering data for ns/openshift-dns...
2026-09-18T20:07:31.513904731Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:31.517302921Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-18T20:07:31.532306035Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-18T20:07:31.693186806Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:31.880725858Z Wrote inspect data to /must-gather/istio.
2026-09-18T20:07:32.154786932Z Gathering data for ns/openshift-image-registry...
2026-09-18T20:07:32.255728881Z Waiting for performance profile collector pods to become ready: 2
2026-09-18T20:07:32.586572360Z Done executing Istio gather script
2026-09-18T20:07:32.662662050Z Gathering data for ns/openshift-ingress-operator...
2026-09-18T20:07:33.022981431Z Gathering data for ns/openshift-ingress...
2026-09-18T20:07:33.348671453Z Gathering data for ns/openshift-ingress-canary...
2026-09-18T20:07:33.379218605Z Waiting for performance profile collector pods to become ready: 3
2026-09-18T20:07:33.725031447Z Gathering data for ns/openshift-insights...
2026-09-18T20:07:34.487320472Z Waiting for performance profile collector pods to become ready: 4
2026-09-18T20:07:35.602083893Z Waiting for performance profile collector pods to become ready: 5
2026-09-18T20:07:36.027616891Z Gathering data for ns/kserve...
2026-09-18T20:07:36.537519833Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-18T20:07:36.712444411Z Waiting for performance profile collector pods to become ready: 6
2026-09-18T20:07:37.823428993Z Gathering data for ns/openshift-config...
2026-09-18T20:07:37.831225565Z Waiting for performance profile collector pods to become ready: 7
2026-09-18T20:07:38.132606220Z Gathering data for ns/openshift-config-managed...
2026-09-18T20:07:38.585326899Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-18T20:07:38.849033977Z Gathering data for ns/openshift-kube-apiserver...
2026-09-18T20:07:38.963567967Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-18T20:07:39.169856608Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-18T20:07:39.297837678Z Collecting performance related data for node ip-10-0-130-199.ec2.internal
2026-09-18T20:07:39.304606631Z Collecting performance related data for node ip-10-0-141-190.ec2.internal
2026-09-18T20:07:39.311631493Z Collecting performance related data for node ip-10-0-143-198.ec2.internal
2026-09-18T20:07:39.542720047Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-18T20:07:39.874752900Z Gathering data for ns/openshift-kube-scheduler...
2026-09-18T20:07:40.216035957Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-18T20:07:40.568787968Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-18T20:07:40.948992650Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-18T20:07:41.259929611Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-18T20:07:41.543984036Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-18T20:07:41.800523539Z Gathering data for ns/openshift-multus...
2026-09-18T20:07:43.220508363Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-18T20:07:44.723333951Z Gathering data for ns/openshift-host-network...
2026-09-18T20:07:45.037881883Z Gathering data for ns/openshift-network-diagnostics...
2026-09-18T20:07:45.558947498Z Gathering data for ns/openshift-network-node-identity...
2026-09-18T20:07:45.947571545Z Gathering data for ns/openshift-network-operator...
2026-09-18T20:07:46.320616279Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-18T20:07:46.550937829Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-09-18T20:07:46.592007211Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-18T20:07:47.102867054Z Gathering data for ns/openshift-apiserver-operator...
2026-09-18T20:07:47.247875023Z Collecting kubelet logs for node ip-10-0-130-199.ec2.internal
2026-09-18T20:07:47.249582944Z Collecting kubelet logs for node ip-10-0-141-190.ec2.internal
2026-09-18T20:07:47.251640452Z Collecting kubelet logs for node ip-10-0-143-198.ec2.internal
2026-09-18T20:07:47.440227090Z Gathering data for ns/openshift-apiserver...
2026-09-18T20:07:47.663609534Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-rlcn6 namespace
2026-09-18T20:07:47.668791502Z INFO: Node performance data collection complete.
2026-09-18T20:07:47.878606858Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-18T20:07:48.180351719Z Gathering data for ns/openshift-controller-manager...
2026-09-18T20:07:48.447202396Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-18T20:07:49.064906352Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-18T20:07:49.337021167Z Gathering data for ns/openshift-service-ca-operator...
2026-09-18T20:07:49.646681906Z Gathering data for ns/openshift-service-ca...
2026-09-18T20:07:49.986077118Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-18T20:07:50.841604993Z Wrote inspect data to must-gather.
2026-09-18T20:07:50.841648850Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T20:07:50.841648850Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-18T20:07:50.971714546Z error: the server doesn't have a resource type "clusters"
2026-09-18T20:07:51.105852896Z Caches written to disk