2026-06-12T21:12:34.922825246Z [disk usage checker] Started
2026-06-12T21:12:34.926052110Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T21:12:35.115326532Z Gathering data for ns/openshift-cluster-version...
2026-06-12T21:12:35.231740660Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-12T21:12:35.315225976Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-12T21:12:35.385829139Z Gathering data for ns/default...
2026-06-12T21:12:35.655612233Z Gathering data for ns/openshift...
2026-06-12T21:12:36.034675734Z Gathering data for ns/kube-system...
2026-06-12T21:12:36.683435477Z Gathering data for ns/openshift-etcd...
2026-06-12T21:12:36.961602151Z Wrote inspect data to must-gather.
2026-06-12T21:12:36.961622115Z error: inspection completed with the errors occurred while gathering data:
2026-06-12T21:12:36.961622115Z     namespaces "assisted-installer" not found
2026-06-12T21:12:37.440192809Z Waiting on subprocesses to finish execution.
2026-06-12T21:12:37.446257850Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-12T21:12:37.450511451Z INFO: Collecting host service logs for crio
2026-06-12T21:12:37.450921008Z INFO: Collecting host service logs for kubelet
2026-06-12T21:12:37.451266059Z INFO: Collecting host service logs for rpm-ostreed
2026-06-12T21:12:37.451601019Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-12T21:12:37.452077980Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-12T21:12:37.452408580Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-12T21:12:37.452737609Z INFO: Collecting host service logs for NetworkManager
2026-06-12T21:12:37.453107545Z INFO: Collecting host service logs for openvswitch
2026-06-12T21:12:37.453440799Z INFO: Collecting host service logs for ovs-configuration
2026-06-12T21:12:37.453786611Z INFO: Collecting host service logs for ovsdb-server
2026-06-12T21:12:37.454121493Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-12T21:12:37.454502733Z INFO: Waiting for worker host service log collection to complete ...
2026-06-12T21:12:37.465856824Z Executing Istio gather script
2026-06-12T21:12:37.474645789Z INFO: Gathering HAProxy config files
2026-06-12T21:12:37.478140604Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-12T21:12:37.544834152Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-12T21:12:37.639390856Z INFO: Waiting for node performance related collection to complete ...
2026-06-12T21:12:37.977784727Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-12T21:12:38.332454736Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-12T21:12:38.387896199Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-12T21:12:38.452532709Z No resources found in openshift-etcd namespace.
2026-06-12T21:12:38.985643922Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-12T21:12:39.015740001Z No resources found
2026-06-12T21:12:39.021100847Z Gathering data for ns/openshift-monitoring...
2026-06-12T21:12:39.098241750Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-12T21:12:39.118213958Z Wrote inspect data to must-gather.
2026-06-12T21:12:39.362706007Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-12T21:12:39.463045698Z INFO: Collecting Insights Archives from insights-operator-5bbd86d6bd-bp72d
2026-06-12T21:12:39.463103478Z insights-runtime-extractor-h2tqj
2026-06-12T21:12:39.463126038Z insights-runtime-extractor-vdcqs
2026-06-12T21:12:39.463145482Z insights-runtime-extractor-z4n4z
2026-06-12T21:12:39.482192939Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-12T21:12:39.620393122Z Wrote inspect data to must-gather.
2026-06-12T21:12:39.666555208Z error: the server doesn't have a resource type "performanceprofile"
2026-06-12T21:12:39.675933754Z tar: Removing leading `/' from member names
2026-06-12T21:12:39.676533458Z Wrote inspect data to must-gather.
2026-06-12T21:12:39.680557794Z INFO: HAProxy config collection complete.
2026-06-12T21:12:39.709428523Z Wrote inspect data to must-gather.
2026-06-12T21:12:39.765209723Z INFO: "metallb-operator" not detected. Skipping.
2026-06-12T21:12:39.785777318Z INFO: Worker host service log collection to complete.
2026-06-12T21:12:39.854498853Z INFO: 'previous-logs' folder not found on ip-10-0-129-101.ec2.internal, skipping...
2026-06-12T21:12:39.869397571Z No resources found
2026-06-12T21:12:39.880451522Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-12T21:12:39.880484604Z INFO: on-disk MachineConfig config collection complete.
2026-06-12T21:12:39.894202690Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-12T21:12:39.894202690Z See 'oc rsync -h' for help and examples
2026-06-12T21:12:39.895534369Z Wrote inspect data to must-gather.
2026-06-12T21:12:39.896772558Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-12T21:12:39.918706720Z ERROR: No running kube-apiserver pods found
2026-06-12T21:12:39.932637968Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T21:12:40.041267562Z Wrote inspect data to must-gather.
2026-06-12T21:12:40.184476000Z Wrote inspect data to must-gather.
2026-06-12T21:12:40.197810135Z Wrote inspect data to must-gather.
2026-06-12T21:12:40.199009720Z INFO: Getting rules from prometheus-k8s-0
2026-06-12T21:12:40.225369404Z INFO: 'previous-logs' folder not found on ip-10-0-132-193.ec2.internal, skipping...
2026-06-12T21:12:40.300807141Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-12T21:12:40.377014232Z error: the server doesn't have a resource type "machineconfigs"
2026-06-12T21:12:40.466921897Z Wrote inspect data to must-gather.
2026-06-12T21:12:40.519570536Z INFO: 'previous-logs' folder not found on ip-10-0-133-14.ec2.internal, skipping...
2026-06-12T21:12:40.519608860Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-12T21:12:40.519632217Z INFO: Machine Config Daemon termination log collection complete.
2026-06-12T21:12:40.546227646Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-12T21:12:40.628052859Z INFO: Getting status/config from prometheus-k8s-0
2026-06-12T21:12:40.710026229Z Wrote inspect data to must-gather.
2026-06-12T21:12:40.771399664Z Wrote inspect data to must-gather.
2026-06-12T21:12:40.947940772Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-12T21:12:41.019274327Z INFO: INTERCONNECT MODE
2026-06-12T21:12:41.019379077Z INFO: Gathering ovn-kubernetes DBs
2026-06-12T21:12:41.048661045Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-12T21:12:41.085919732Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-12T21:12:41.156376817Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-12T21:12:41.169271841Z Wrote inspect data to must-gather.
2026-06-12T21:12:41.199331657Z INFO: Gathering OVN_Northbound from ovnkube-node-9thf2...
2026-06-12T21:12:41.199654689Z INFO: Gathering OVN_Northbound from ovnkube-node-d5xgd...
2026-06-12T21:12:41.199977337Z INFO: Gathering OVN_Northbound from ovnkube-node-lnntf...
2026-06-12T21:12:41.200297547Z INFO: Gathering OVN_Southbound from ovnkube-node-9thf2...
2026-06-12T21:12:41.200593325Z INFO: Gathering OVN_Southbound from ovnkube-node-d5xgd...
2026-06-12T21:12:41.200898901Z INFO: Gathering OVN_Southbound from ovnkube-node-lnntf...
2026-06-12T21:12:41.471529334Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-12T21:12:41.494594765Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:41.596368109Z tar: Removing leading `/' from member names
2026-06-12T21:12:41.599037895Z tar: Removing leading `/' from member names
2026-06-12T21:12:41.610737657Z Wrote inspect data to must-gather.
2026-06-12T21:12:41.614445921Z tar: Removing leading `/' from member names
2026-06-12T21:12:41.651776940Z tar: Removing leading `/' from member names
2026-06-12T21:12:41.653574970Z tar: Removing leading `/' from member names
2026-06-12T21:12:41.657146436Z tar: Removing leading `/' from member names
2026-06-12T21:12:41.786349505Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:41.788928253Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-12T21:12:41.799988550Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-12T21:12:41.873840562Z W0612 21:12:41.873784    1399 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-12T21:12:41.880344793Z Wrote inspect data to must-gather.
2026-06-12T21:12:41.993141440Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-12T21:12:42.023556490Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:42.070218393Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-12T21:12:42.131130369Z INFO: Waiting for network log collection to complete ...
2026-06-12T21:12:42.131251204Z INFO: Waiting for ovnk database copies to complete ...
2026-06-12T21:12:42.131872417Z INFO: Copying ovnk databases complete.
2026-06-12T21:12:42.133847388Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-06-12T21:12:42.136101419Z ovnk_database_store/
2026-06-12T21:12:42.136101419Z ovnk_database_store/ovnkube-node-d5xgd_sbdb
2026-06-12T21:12:42.156223255Z ovnk_database_store/ovnkube-node-d5xgd_nbdb
2026-06-12T21:12:42.162227667Z ovnk_database_store/ovnkube-node-lnntf_sbdb
2026-06-12T21:12:42.179472056Z ovnk_database_store/ovnkube-node-9thf2_nbdb
2026-06-12T21:12:42.188808414Z ovnk_database_store/ovnkube-node-9thf2_sbdb
2026-06-12T21:12:42.205839185Z ovnk_database_store/ovnkube-node-lnntf_nbdb
2026-06-12T21:12:42.217147321Z INFO: Network log collection complete.
2026-06-12T21:12:42.221461205Z Gathering data for ns/openshift-network-console...
2026-06-12T21:12:42.229674878Z Wrote inspect data to must-gather.
2026-06-12T21:12:42.262287290Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:42.265878652Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-12T21:12:42.485965586Z INFO: Getting status from alertmanager-main-0
2026-06-12T21:12:42.487119995Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:42.545788446Z Wrote inspect data to must-gather.
2026-06-12T21:12:42.695880531Z Gathering data for ns/openshift-console-operator...
2026-06-12T21:12:42.709524741Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:42.712309526Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-12T21:12:42.856327479Z Wrote inspect data to must-gather.
2026-06-12T21:12:42.926233047Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:43.104308759Z Gathering data for ns/openshift-console...
2026-06-12T21:12:43.131591963Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:43.133837057Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-12T21:12:43.167490489Z Wrote inspect data to must-gather.
2026-06-12T21:12:43.311362871Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:43.426918440Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-12T21:12:43.429812029Z INFO: Fallback to identify the container image from release info
2026-06-12T21:12:43.491134782Z Wrote inspect data to /must-gather/istio.
2026-06-12T21:12:43.656882851Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-12T21:12:44.010172417Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d66a6b2ec52dede5d83ce04f6abfc519df1fd7dc36248741e03ecb9bf8d1762
2026-06-12T21:12:44.144874270Z daemonset.apps/perf-node-gather-daemonset created
2026-06-12T21:12:44.191159414Z Done executing Istio gather script
2026-06-12T21:12:44.194336113Z Gathering data for ns/openshift-dns-operator...
2026-06-12T21:12:44.286715985Z Waiting for performance profile collector pods to become ready: 1
2026-06-12T21:12:44.473965508Z Gathering data for ns/openshift-dns...
2026-06-12T21:12:44.938143364Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T21:12:45.052261597Z Gathering data for ns/openshift-image-registry...
2026-06-12T21:12:45.399702131Z Waiting for performance profile collector pods to become ready: 2
2026-06-12T21:12:45.490097948Z Gathering data for ns/openshift-ingress-operator...
2026-06-12T21:12:45.782850927Z Gathering data for ns/openshift-ingress...
2026-06-12T21:12:46.080179115Z Gathering data for ns/openshift-ingress-canary...
2026-06-12T21:12:46.433276560Z Gathering data for ns/openshift-insights...
2026-06-12T21:12:46.508597423Z Waiting for performance profile collector pods to become ready: 3
2026-06-12T21:12:47.612330343Z Waiting for performance profile collector pods to become ready: 4
2026-06-12T21:12:48.237923253Z Gathering data for ns/openshift-config...
2026-06-12T21:12:48.520403207Z Gathering data for ns/openshift-config-managed...
2026-06-12T21:12:48.723665139Z Waiting for performance profile collector pods to become ready: 5
2026-06-12T21:12:48.910661829Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-12T21:12:49.182778892Z Gathering data for ns/openshift-kube-apiserver...
2026-06-12T21:12:49.422266770Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-12T21:12:49.683597795Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-12T21:12:49.841577495Z Waiting for performance profile collector pods to become ready: 6
2026-06-12T21:12:49.931143108Z Gathering data for ns/openshift-kube-scheduler...
2026-06-12T21:12:49.945200932Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T21:12:50.180222377Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-12T21:12:50.424279464Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-12T21:12:50.731435738Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-12T21:12:50.953678351Z Waiting for performance profile collector pods to become ready: 7
2026-06-12T21:12:51.017820946Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-12T21:12:51.450046247Z Gathering data for ns/openshift-multus...
2026-06-12T21:12:52.055226344Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-12T21:12:52.296324813Z Collecting performance related data for node ip-10-0-132-193.ec2.internal
2026-06-12T21:12:52.306159703Z Collecting performance related data for node ip-10-0-133-14.ec2.internal
2026-06-12T21:12:52.314557266Z Collecting performance related data for node ip-10-0-129-101.ec2.internal
2026-06-12T21:12:52.795860805Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-12T21:12:54.076618013Z Gathering data for ns/openshift-host-network...
2026-06-12T21:12:54.362915308Z Gathering data for ns/openshift-network-diagnostics...
2026-06-12T21:12:54.896450734Z Gathering data for ns/openshift-network-node-identity...
2026-06-12T21:12:54.952049450Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T21:12:55.314042925Z Gathering data for ns/openshift-network-operator...
2026-06-12T21:12:55.709013946Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-12T21:12:55.940248991Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-12T21:12:56.395580397Z Gathering data for ns/openshift-apiserver-operator...
2026-06-12T21:12:56.685079767Z Gathering data for ns/openshift-apiserver...
2026-06-12T21:12:57.034926642Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-12T21:12:57.326738385Z Gathering data for ns/openshift-controller-manager...
2026-06-12T21:12:57.582290050Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-12T21:12:58.204497297Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-12T21:12:58.474167269Z Gathering data for ns/openshift-service-ca-operator...
2026-06-12T21:12:58.756041376Z Gathering data for ns/openshift-service-ca...
2026-06-12T21:12:59.113000567Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-12T21:12:59.958477245Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-12T21:12:59.988909024Z Wrote inspect data to must-gather.
2026-06-12T21:12:59.988955294Z error: inspection completed with the errors occurred while gathering data:
2026-06-12T21:12:59.988955294Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-12T21:13:00.440820540Z Collecting kubelet logs for node ip-10-0-132-193.ec2.internal
2026-06-12T21:13:00.442566129Z Collecting kubelet logs for node ip-10-0-133-14.ec2.internal
2026-06-12T21:13:00.444333134Z Collecting kubelet logs for node ip-10-0-129-101.ec2.internal
2026-06-12T21:13:00.767244131Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-srwvb namespace
2026-06-12T21:13:00.773802555Z INFO: Node performance data collection complete.
2026-06-12T21:13:00.899622517Z error: the server doesn't have a resource type "clusters"
2026-06-12T21:13:01.024908781Z Caches written to disk