2026-09-11T06:31:23.725446526Z [disk usage checker] Started
2026-09-11T06:31:23.729577268Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-11T06:31:23.933125886Z Gathering data for ns/openshift-cluster-version...
2026-09-11T06:31:24.070039542Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T06:31:24.160777723Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T06:31:24.243606741Z Gathering data for ns/default...
2026-09-11T06:31:24.548553444Z Gathering data for ns/openshift...
2026-09-11T06:31:24.947913010Z Gathering data for ns/kube-system...
2026-09-11T06:31:25.595720161Z Gathering data for ns/openshift-etcd...
2026-09-11T06:31:25.880879002Z Wrote inspect data to must-gather.
2026-09-11T06:31:25.880972493Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T06:31:25.880972493Z     namespaces "assisted-installer" not found
2026-09-11T06:31:26.482963865Z Waiting on subprocesses to finish execution.
2026-09-11T06:31:26.487511966Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-11T06:31:26.492365590Z INFO: Gathering HAProxy config files
2026-09-11T06:31:26.494251737Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-11T06:31:26.515833957Z Executing Istio gather script
2026-09-11T06:31:26.518471783Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-11T06:31:26.556607147Z INFO: Collecting host service logs for crio
2026-09-11T06:31:26.557028168Z INFO: Collecting host service logs for kubelet
2026-09-11T06:31:26.557435431Z INFO: Collecting host service logs for rpm-ostreed
2026-09-11T06:31:26.557748867Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-11T06:31:26.558058148Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-11T06:31:26.558352700Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-11T06:31:26.558696403Z INFO: Collecting host service logs for NetworkManager
2026-09-11T06:31:26.559003634Z INFO: Collecting host service logs for openvswitch
2026-09-11T06:31:26.559296593Z INFO: Collecting host service logs for ovs-configuration
2026-09-11T06:31:26.559637124Z INFO: Collecting host service logs for ovsdb-server
2026-09-11T06:31:26.559939241Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-11T06:31:26.560301002Z INFO: Waiting for worker host service log collection to complete ...
2026-09-11T06:31:26.582377046Z INFO: Waiting for node performance related collection to complete ...
2026-09-11T06:31:27.273676262Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-11T06:31:27.369763799Z No resources found
2026-09-11T06:31:27.372434855Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-11T06:31:27.408095909Z Wrote inspect data to must-gather.
2026-09-11T06:31:27.434602206Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-11T06:31:27.534389048Z INFO: "metallb-operator" not detected. Skipping.
2026-09-11T06:31:27.700436290Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-11T06:31:27.700513262Z INFO: on-disk MachineConfig config collection complete.
2026-09-11T06:31:27.879079964Z ERROR: No running kube-apiserver pods found
2026-09-11T06:31:27.904119354Z INFO: Collecting Insights Archives from insights-operator-5844b856db-2f8zx
2026-09-11T06:31:27.904119354Z insights-runtime-extractor-9bcsl
2026-09-11T06:31:27.904119354Z insights-runtime-extractor-9phtx
2026-09-11T06:31:27.904119354Z insights-runtime-extractor-j52z8
2026-09-11T06:31:27.912247474Z No resources found in openshift-etcd namespace.
2026-09-11T06:31:27.975324275Z error: the server doesn't have a resource type "performanceprofile"
2026-09-11T06:31:28.007700886Z INFO: Worker host service log collection to complete.
2026-09-11T06:31:28.049216557Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-11T06:31:28.267742968Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-11T06:31:28.267742968Z See 'oc rsync -h' for help and examples
2026-09-11T06:31:28.445490973Z Wrote inspect data to must-gather.
2026-09-11T06:31:28.620183638Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-11T06:31:28.626567339Z No resources found
2026-09-11T06:31:28.690172441Z Wrote inspect data to must-gather.
2026-09-11T06:31:28.702804281Z Wrote inspect data to must-gather.
2026-09-11T06:31:28.732723013Z Gathering data for ns/openshift-monitoring...
2026-09-11T06:31:28.880463073Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-11T06:31:28.948325151Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T06:31:29.055947190Z Wrote inspect data to must-gather.
2026-09-11T06:31:29.062689307Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-11T06:31:29.161902538Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-11T06:31:29.203200606Z error: the server doesn't have a resource type "machineconfigs"
2026-09-11T06:31:29.229950948Z Wrote inspect data to must-gather.
2026-09-11T06:31:29.249091302Z INFO: 'previous-logs' folder not found on ip-10-0-132-173.ec2.internal, skipping...
2026-09-11T06:31:29.289738693Z Wrote inspect data to must-gather.
2026-09-11T06:31:29.358802826Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-11T06:31:29.443723366Z INFO: Getting rules from prometheus-k8s-0
2026-09-11T06:31:29.445992374Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-11T06:31:29.466612027Z tar: Removing leading `/' from member names
2026-09-11T06:31:29.471719267Z INFO: HAProxy config collection complete.
2026-09-11T06:31:29.544462236Z Wrote inspect data to must-gather.
2026-09-11T06:31:29.597740452Z INFO: 'previous-logs' folder not found on ip-10-0-134-58.ec2.internal, skipping...
2026-09-11T06:31:29.704394770Z Wrote inspect data to must-gather.
2026-09-11T06:31:29.830933488Z Wrote inspect data to must-gather.
2026-09-11T06:31:29.899708981Z INFO: 'previous-logs' folder not found on ip-10-0-137-116.ec2.internal, skipping...
2026-09-11T06:31:29.899745492Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-11T06:31:29.899768771Z INFO: Machine Config Daemon termination log collection complete.
2026-09-11T06:31:29.908167134Z INFO: Getting status/config from prometheus-k8s-0
2026-09-11T06:31:29.909293365Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-11T06:31:30.108139385Z Wrote inspect data to must-gather.
2026-09-11T06:31:30.147763522Z Wrote inspect data to must-gather.
2026-09-11T06:31:30.389481623Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-11T06:31:30.407166175Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-11T06:31:30.415958900Z Wrote inspect data to must-gather.
2026-09-11T06:31:30.449236901Z INFO: INTERCONNECT MODE
2026-09-11T06:31:30.449303259Z INFO: Gathering ovn-kubernetes DBs
2026-09-11T06:31:30.633310968Z INFO: Gathering OVN_Northbound from ovnkube-node-2h6sh...
2026-09-11T06:31:30.633696001Z INFO: Gathering OVN_Northbound from ovnkube-node-cm6cn...
2026-09-11T06:31:30.633996506Z INFO: Gathering OVN_Northbound from ovnkube-node-rs5f4...
2026-09-11T06:31:30.634269217Z INFO: Gathering OVN_Southbound from ovnkube-node-2h6sh...
2026-09-11T06:31:30.634711687Z INFO: Gathering OVN_Southbound from ovnkube-node-cm6cn...
2026-09-11T06:31:30.635033972Z INFO: Gathering OVN_Southbound from ovnkube-node-rs5f4...
2026-09-11T06:31:30.684839840Z Wrote inspect data to must-gather.
2026-09-11T06:31:30.704027298Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-11T06:31:30.850467017Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-11T06:31:30.984543274Z tar: Removing leading `/' from member names
2026-09-11T06:31:30.991945641Z tar: Removing leading `/' from member names
2026-09-11T06:31:30.992140168Z tar: Removing leading `/' from member names
2026-09-11T06:31:31.008262394Z tar: Removing leading `/' from member names
2026-09-11T06:31:31.060690962Z tar: Removing leading `/' from member names
2026-09-11T06:31:31.080806267Z W0911 06:31:31.080746    1340 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-11T06:31:31.088638681Z Wrote inspect data to must-gather.
2026-09-11T06:31:31.091086103Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-11T06:31:31.103665386Z tar: Removing leading `/' from member names
2026-09-11T06:31:31.171981896Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:31.197782802Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-11T06:31:31.395130834Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-11T06:31:31.439540863Z Gathering data for ns/openshift-network-console...
2026-09-11T06:31:31.470438575Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:31.474246260Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-11T06:31:31.487121963Z Wrote inspect data to must-gather.
2026-09-11T06:31:31.696317453Z INFO: Waiting for network log collection to complete ...
2026-09-11T06:31:31.696355017Z INFO: Waiting for ovnk database copies to complete ...
2026-09-11T06:31:31.696883280Z INFO: Copying ovnk databases complete.
2026-09-11T06:31:31.699009585Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-11T06:31:31.704545975Z ovnk_database_store/
2026-09-11T06:31:31.704609061Z ovnk_database_store/ovnkube-node-cm6cn_nbdb
2026-09-11T06:31:31.716779963Z ovnk_database_store/ovnkube-node-2h6sh_sbdb
2026-09-11T06:31:31.729334532Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:31.734674870Z ovnk_database_store/ovnkube-node-2h6sh_nbdb
2026-09-11T06:31:31.740574371Z ovnk_database_store/ovnkube-node-rs5f4_nbdb
2026-09-11T06:31:31.748904275Z ovnk_database_store/ovnkube-node-cm6cn_sbdb
2026-09-11T06:31:31.769978441Z ovnk_database_store/ovnkube-node-rs5f4_sbdb
2026-09-11T06:31:31.795703132Z INFO: Network log collection complete.
2026-09-11T06:31:31.876919622Z Wrote inspect data to must-gather.
2026-09-11T06:31:31.894828457Z INFO: Getting status from alertmanager-main-0
2026-09-11T06:31:31.986613864Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:31.990121433Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-11T06:31:31.990239512Z Gathering data for ns/openshift-console-operator...
2026-09-11T06:31:32.208121776Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:32.213894107Z Wrote inspect data to must-gather.
2026-09-11T06:31:32.414368932Z Gathering data for ns/openshift-console...
2026-09-11T06:31:32.440025091Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:32.442654161Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-11T06:31:32.499811541Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-11T06:31:32.502728603Z INFO: Fallback to identify the container image from release info
2026-09-11T06:31:32.681107672Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:32.890399410Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:32.892922469Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-11T06:31:32.956013035Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-11T06:31:33.067924607Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:33.189364401Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-11T06:31:33.295165581Z Wrote inspect data to /must-gather/istio.
2026-09-11T06:31:33.333018441Z daemonset.apps/perf-node-gather-daemonset created
2026-09-11T06:31:33.493252402Z Waiting for performance profile collector pods to become ready: 1
2026-09-11T06:31:33.512937467Z Gathering data for ns/openshift-dns-operator...
2026-09-11T06:31:33.826291422Z Gathering data for ns/openshift-dns...
2026-09-11T06:31:33.888075748Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-11T06:31:34.046283358Z Done executing Istio gather script
2026-09-11T06:31:34.414316244Z Gathering data for ns/openshift-image-registry...
2026-09-11T06:31:34.621672928Z Waiting for performance profile collector pods to become ready: 2
2026-09-11T06:31:34.859908588Z Gathering data for ns/openshift-ingress-operator...
2026-09-11T06:31:35.155762030Z Gathering data for ns/openshift-ingress...
2026-09-11T06:31:35.465107041Z Gathering data for ns/openshift-ingress-canary...
2026-09-11T06:31:35.748318681Z Waiting for performance profile collector pods to become ready: 3
2026-09-11T06:31:35.833007755Z Gathering data for ns/openshift-insights...
2026-09-11T06:31:36.866625946Z Waiting for performance profile collector pods to become ready: 4
2026-09-11T06:31:37.645969560Z Gathering data for ns/openshift-jobset-operator...
2026-09-11T06:31:37.996787878Z Waiting for performance profile collector pods to become ready: 5
2026-09-11T06:31:38.462095359Z Gathering data for ns/openshift-config...
2026-09-11T06:31:38.700956437Z Gathering data for ns/openshift-config-managed...
2026-09-11T06:31:38.893364663Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-11T06:31:39.125388043Z Waiting for performance profile collector pods to become ready: 6
2026-09-11T06:31:39.127294103Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-11T06:31:39.410174944Z Gathering data for ns/openshift-kube-apiserver...
2026-09-11T06:31:39.647530004Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-11T06:31:39.913227058Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-11T06:31:40.179809615Z Gathering data for ns/openshift-kube-scheduler...
2026-09-11T06:31:40.270944571Z Waiting for performance profile collector pods to become ready: 7
2026-09-11T06:31:40.432380533Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-11T06:31:40.710240323Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-11T06:31:41.035382902Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-11T06:31:41.345439666Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-11T06:31:41.409182689Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-11T06:31:41.676994484Z Collecting performance related data for node ip-10-0-132-173.ec2.internal
2026-09-11T06:31:41.684986775Z Collecting performance related data for node ip-10-0-137-116.ec2.internal
2026-09-11T06:31:41.694151817Z Collecting performance related data for node ip-10-0-134-58.ec2.internal
2026-09-11T06:31:41.898124341Z Gathering data for ns/openshift-multus...
2026-09-11T06:31:43.420134199Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-11T06:31:43.899162596Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-11T06:31:44.622774772Z Gathering data for ns/openshift-host-network...
2026-09-11T06:31:44.938293153Z Gathering data for ns/openshift-network-diagnostics...
2026-09-11T06:31:45.451574332Z Gathering data for ns/openshift-network-node-identity...
2026-09-11T06:31:45.881341762Z Gathering data for ns/openshift-network-operator...
2026-09-11T06:31:46.253802880Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-11T06:31:46.510481403Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-11T06:31:46.975642143Z Gathering data for ns/openshift-apiserver-operator...
2026-09-11T06:31:47.263067171Z Gathering data for ns/openshift-apiserver...
2026-09-11T06:31:47.741023820Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-11T06:31:47.992571047Z Gathering data for ns/openshift-controller-manager...
2026-09-11T06:31:48.241264625Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-11T06:31:48.891914946Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-11T06:31:48.907221190Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-11T06:31:49.162700259Z Gathering data for ns/openshift-service-ca-operator...
2026-09-11T06:31:49.449996114Z Gathering data for ns/openshift-service-ca...
2026-09-11T06:31:49.799659477Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-11T06:31:49.976706902Z Collecting kubelet logs for node ip-10-0-132-173.ec2.internal
2026-09-11T06:31:49.978744454Z Collecting kubelet logs for node ip-10-0-137-116.ec2.internal
2026-09-11T06:31:49.980917148Z Collecting kubelet logs for node ip-10-0-134-58.ec2.internal
2026-09-11T06:31:50.367338296Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4sbc5 namespace
2026-09-11T06:31:50.372379377Z INFO: Node performance data collection complete.
2026-09-11T06:31:50.778277233Z Wrote inspect data to must-gather.
2026-09-11T06:31:50.778328530Z error: inspection completed with the errors occurred while gathering data:
2026-09-11T06:31:50.778328530Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-11T06:31:50.915642408Z error: the server doesn't have a resource type "clusters"
2026-09-11T06:31:51.053833420Z Caches written to disk