2026-07-06T21:06:22.812013020Z [disk usage checker] Started
2026-07-06T21:06:22.815342330Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T21:06:23.011299105Z Gathering data for ns/openshift-cluster-version...
2026-07-06T21:06:23.128038612Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T21:06:23.235951968Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T21:06:23.321285141Z Gathering data for ns/default...
2026-07-06T21:06:23.744262022Z Gathering data for ns/openshift...
2026-07-06T21:06:24.154038972Z Gathering data for ns/kube-system...
2026-07-06T21:06:24.883763803Z Gathering data for ns/openshift-etcd...
2026-07-06T21:06:25.224931508Z Wrote inspect data to must-gather.
2026-07-06T21:06:25.224965443Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T21:06:25.224965443Z     namespaces "assisted-installer" not found
2026-07-06T21:06:25.309028324Z Waiting on subprocesses to finish execution.
2026-07-06T21:06:25.318321140Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-06T21:06:25.318730594Z INFO: Collecting host service logs for crio
2026-07-06T21:06:25.319124219Z INFO: Collecting host service logs for kubelet
2026-07-06T21:06:25.319465098Z INFO: Collecting host service logs for rpm-ostreed
2026-07-06T21:06:25.319806410Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-06T21:06:25.320149434Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-06T21:06:25.320499103Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-06T21:06:25.320842960Z INFO: Collecting host service logs for NetworkManager
2026-07-06T21:06:25.321200156Z INFO: Collecting host service logs for openvswitch
2026-07-06T21:06:25.321557275Z INFO: Collecting host service logs for ovs-configuration
2026-07-06T21:06:25.321596633Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-06T21:06:25.321918103Z INFO: Collecting host service logs for ovsdb-server
2026-07-06T21:06:25.322366339Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-06T21:06:25.322751004Z INFO: Waiting for worker host service log collection to complete ...
2026-07-06T21:06:25.324427336Z INFO: Gathering HAProxy config files
2026-07-06T21:06:25.325645016Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-06T21:06:25.337868361Z Executing Istio gather script
2026-07-06T21:06:25.495920367Z INFO: Waiting for node performance related collection to complete ...
2026-07-06T21:06:25.999253935Z INFO: Collecting Insights Archives from insights-operator-5c847dcfc4-kt2zx
2026-07-06T21:06:25.999301886Z insights-runtime-extractor-76md2
2026-07-06T21:06:25.999321906Z insights-runtime-extractor-lmjcx
2026-07-06T21:06:25.999341204Z insights-runtime-extractor-zj2hr
2026-07-06T21:06:26.142136050Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-06T21:06:26.179933307Z No resources found
2026-07-06T21:06:26.280434215Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-06T21:06:26.292552771Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-06T21:06:26.414820737Z Wrote inspect data to must-gather.
2026-07-06T21:06:26.467498374Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-06T21:06:26.536406949Z No resources found in openshift-etcd namespace.
2026-07-06T21:06:26.622189469Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-06T21:06:26.675164566Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-06T21:06:26.702718673Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-06T21:06:26.702718673Z See 'oc rsync -h' for help and examples
2026-07-06T21:06:26.709698983Z error: the server doesn't have a resource type "performanceprofile"
2026-07-06T21:06:26.758012625Z Wrote inspect data to must-gather.
2026-07-06T21:06:26.854949008Z INFO: "metallb-operator" not detected. Skipping.
2026-07-06T21:06:26.871602960Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-06T21:06:26.871679829Z INFO: on-disk MachineConfig config collection complete.
2026-07-06T21:06:27.198765768Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-06T21:06:27.247127127Z ERROR: No running kube-apiserver pods found
2026-07-06T21:06:27.282539641Z INFO: HAProxy config collection complete.
2026-07-06T21:06:27.292854976Z INFO: Worker host service log collection to complete.
2026-07-06T21:06:27.443129665Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T21:06:27.502996858Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-06T21:06:27.562904476Z Wrote inspect data to must-gather.
2026-07-06T21:06:27.614393749Z Wrote inspect data to must-gather.
2026-07-06T21:06:27.664601073Z Wrote inspect data to must-gather.
2026-07-06T21:06:27.742104333Z Wrote inspect data to must-gather.
2026-07-06T21:06:27.742421853Z No resources found
2026-07-06T21:06:27.753919327Z Gathering data for ns/openshift-monitoring...
2026-07-06T21:06:27.778626956Z Wrote inspect data to must-gather.
2026-07-06T21:06:27.816585318Z INFO: Getting rules from prometheus-k8s-0
2026-07-06T21:06:27.825443486Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T21:06:27.847796847Z INFO: 'previous-logs' folder not found on ip-10-0-135-237.ec2.internal, skipping...
2026-07-06T21:06:27.857805343Z error: the server doesn't have a resource type "machineconfigs"
2026-07-06T21:06:28.034736263Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-06T21:06:28.051339328Z Wrote inspect data to must-gather.
2026-07-06T21:06:28.127844362Z INFO: 'previous-logs' folder not found on ip-10-0-136-222.ec2.internal, skipping...
2026-07-06T21:06:28.244437201Z INFO: Getting status/config from prometheus-k8s-0
2026-07-06T21:06:28.271049506Z Wrote inspect data to must-gather.
2026-07-06T21:06:28.325362398Z Wrote inspect data to must-gather.
2026-07-06T21:06:28.397042668Z INFO: 'previous-logs' folder not found on ip-10-0-139-226.ec2.internal, skipping...
2026-07-06T21:06:28.397163383Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-06T21:06:28.397221586Z INFO: Machine Config Daemon termination log collection complete.
2026-07-06T21:06:28.441803240Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-06T21:06:28.571117150Z Wrote inspect data to must-gather.
2026-07-06T21:06:28.646462976Z Wrote inspect data to must-gather.
2026-07-06T21:06:28.690386874Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-06T21:06:28.775426439Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-06T21:06:28.843785823Z Wrote inspect data to must-gather.
2026-07-06T21:06:28.844865054Z INFO: INTERCONNECT MODE
2026-07-06T21:06:28.844893249Z INFO: Gathering ovn-kubernetes DBs
2026-07-06T21:06:28.918165185Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-06T21:06:28.945887880Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:29.007879772Z INFO: Gathering OVN_Northbound from ovnkube-node-9wln2...
2026-07-06T21:06:29.008132094Z INFO: Gathering OVN_Northbound from ovnkube-node-q2cbc...
2026-07-06T21:06:29.008379350Z INFO: Gathering OVN_Northbound from ovnkube-node-zdrfx...
2026-07-06T21:06:29.008663051Z INFO: Gathering OVN_Southbound from ovnkube-node-9wln2...
2026-07-06T21:06:29.009282925Z INFO: Gathering OVN_Southbound from ovnkube-node-q2cbc...
2026-07-06T21:06:29.009585831Z INFO: Gathering OVN_Southbound from ovnkube-node-zdrfx...
2026-07-06T21:06:29.075486270Z Wrote inspect data to must-gather.
2026-07-06T21:06:29.230423264Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-06T21:06:29.274448943Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:29.278171071Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-06T21:06:29.335764906Z tar: Removing leading `/' from member names
2026-07-06T21:06:29.335800892Z tar: Removing leading `/' from member names
2026-07-06T21:06:29.344234995Z tar: Removing leading `/' from member names
2026-07-06T21:06:29.348489015Z tar: Removing leading `/' from member names
2026-07-06T21:06:29.356606681Z tar: Removing leading `/' from member names
2026-07-06T21:06:29.357168726Z tar: Removing leading `/' from member names
2026-07-06T21:06:29.423598901Z Wrote inspect data to must-gather.
2026-07-06T21:06:29.521415497Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:29.524274494Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-06T21:06:29.526065879Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-06T21:06:29.663514582Z Wrote inspect data to must-gather.
2026-07-06T21:06:29.752007201Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:29.755641856Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-06T21:06:29.862691579Z INFO: Waiting for network log collection to complete ...
2026-07-06T21:06:29.862729693Z INFO: Waiting for ovnk database copies to complete ...
2026-07-06T21:06:29.863112850Z INFO: Copying ovnk databases complete.
2026-07-06T21:06:29.864826076Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-06T21:06:29.867594917Z ovnk_database_store/
2026-07-06T21:06:29.867646056Z ovnk_database_store/ovnkube-node-zdrfx_sbdb
2026-07-06T21:06:29.889689022Z ovnk_database_store/ovnkube-node-q2cbc_sbdb
2026-07-06T21:06:29.900178075Z Wrote inspect data to must-gather.
2026-07-06T21:06:29.906826933Z ovnk_database_store/ovnkube-node-zdrfx_nbdb
2026-07-06T21:06:29.913964061Z ovnk_database_store/ovnkube-node-9wln2_nbdb
2026-07-06T21:06:29.921734317Z ovnk_database_store/ovnkube-node-q2cbc_nbdb
2026-07-06T21:06:29.929235121Z ovnk_database_store/ovnkube-node-9wln2_sbdb
2026-07-06T21:06:29.950825142Z INFO: Getting status from alertmanager-main-0
2026-07-06T21:06:29.952360223Z INFO: Network log collection complete.
2026-07-06T21:06:29.980793619Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:30.106347437Z W0706 21:06:30.106304    1541 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-06T21:06:30.111471672Z Wrote inspect data to must-gather.
2026-07-06T21:06:30.185136247Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:30.187868237Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-06T21:06:30.188259952Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-06T21:06:30.359197193Z Gathering data for ns/openshift-network-console...
2026-07-06T21:06:30.375258539Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-06T21:06:30.377032898Z INFO: Fallback to identify the container image from release info
2026-07-06T21:06:30.406218526Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:30.598778750Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:30.600884344Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-06T21:06:30.757672957Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:30.941337932Z Gathering data for ns/openshift-console-operator...
2026-07-06T21:06:30.950102971Z Wrote inspect data to /must-gather/istio.
2026-07-06T21:06:31.029148121Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:5be8cdcf34f35a89a70b6bfa49e66897641871237570b3e09766f83a0a473e6a
2026-07-06T21:06:31.163824518Z daemonset.apps/perf-node-gather-daemonset created
2026-07-06T21:06:31.296732698Z Waiting for performance profile collector pods to become ready: 1
2026-07-06T21:06:31.411539923Z Gathering data for ns/openshift-console...
2026-07-06T21:06:31.623656417Z Done executing Istio gather script
2026-07-06T21:06:31.922529616Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-06T21:06:32.384555713Z Gathering data for ns/openshift-dns-operator...
2026-07-06T21:06:32.423106887Z Waiting for performance profile collector pods to become ready: 2
2026-07-06T21:06:32.750684881Z Gathering data for ns/openshift-dns...
2026-07-06T21:06:32.829733506Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T21:06:33.354546163Z Gathering data for ns/openshift-image-registry...
2026-07-06T21:06:33.537181117Z Waiting for performance profile collector pods to become ready: 3
2026-07-06T21:06:33.847553210Z Gathering data for ns/openshift-ingress-operator...
2026-07-06T21:06:34.162722915Z Gathering data for ns/openshift-ingress...
2026-07-06T21:06:34.493088141Z Gathering data for ns/openshift-ingress-canary...
2026-07-06T21:06:34.659266078Z Waiting for performance profile collector pods to become ready: 4
2026-07-06T21:06:34.894141808Z Gathering data for ns/openshift-insights...
2026-07-06T21:06:35.772836441Z Waiting for performance profile collector pods to become ready: 5
2026-07-06T21:06:36.877231602Z Waiting for performance profile collector pods to become ready: 6
2026-07-06T21:06:36.878691366Z Gathering data for ns/openshift-config...
2026-07-06T21:06:37.236536464Z Gathering data for ns/openshift-config-managed...
2026-07-06T21:06:37.683226052Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-06T21:06:37.833979725Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T21:06:37.997736681Z Waiting for performance profile collector pods to become ready: 7
2026-07-06T21:06:38.016895477Z Gathering data for ns/openshift-kube-apiserver...
2026-07-06T21:06:38.277346150Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-06T21:06:38.554878644Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-06T21:06:38.854693546Z Gathering data for ns/openshift-kube-scheduler...
2026-07-06T21:06:39.121175659Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-06T21:06:39.128132344Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-06T21:06:39.369940244Z Collecting performance related data for node ip-10-0-139-226.ec2.internal
2026-07-06T21:06:39.377498563Z Collecting performance related data for node ip-10-0-136-222.ec2.internal
2026-07-06T21:06:39.385878558Z Collecting performance related data for node ip-10-0-135-237.ec2.internal
2026-07-06T21:06:39.474734260Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-06T21:06:39.896399259Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-06T21:06:40.267558946Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-06T21:06:40.939967002Z Gathering data for ns/openshift-multus...
2026-07-06T21:06:42.428567521Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-06T21:06:42.838298236Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T21:06:43.878673726Z Gathering data for ns/openshift-host-network...
2026-07-06T21:06:44.326530867Z Gathering data for ns/openshift-network-diagnostics...
2026-07-06T21:06:45.022810920Z Gathering data for ns/openshift-network-node-identity...
2026-07-06T21:06:45.506648828Z Gathering data for ns/openshift-network-operator...
2026-07-06T21:06:45.980933717Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-06T21:06:46.275224887Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-06T21:06:46.776815867Z Gathering data for ns/openshift-apiserver-operator...
2026-07-06T21:06:47.130215077Z Gathering data for ns/openshift-apiserver...
2026-07-06T21:06:47.438707066Z Collecting kubelet logs for node ip-10-0-139-226.ec2.internal
2026-07-06T21:06:47.440443078Z Collecting kubelet logs for node ip-10-0-136-222.ec2.internal
2026-07-06T21:06:47.442899853Z Collecting kubelet logs for node ip-10-0-135-237.ec2.internal
2026-07-06T21:06:47.668015315Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-06T21:06:47.788200336Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-d4w8c namespace
2026-07-06T21:06:47.792955818Z INFO: Node performance data collection complete.
2026-07-06T21:06:47.842613570Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-06T21:06:48.072877994Z Gathering data for ns/openshift-controller-manager...
2026-07-06T21:06:48.381378173Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-06T21:06:49.061788100Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-06T21:06:49.329320334Z Gathering data for ns/openshift-service-ca-operator...
2026-07-06T21:06:49.649885103Z Gathering data for ns/openshift-service-ca...
2026-07-06T21:06:50.069601168Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-06T21:06:50.988465354Z Wrote inspect data to must-gather.
2026-07-06T21:06:50.988509847Z error: inspection completed with the errors occurred while gathering data:
2026-07-06T21:06:50.988509847Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-06T21:06:51.107486059Z error: the server doesn't have a resource type "clusters"
2026-07-06T21:06:51.332055502Z Caches written to disk