2026-09-18T17:43:04.252971091Z [disk usage checker] Started
2026-09-18T17:43:04.256280471Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T17:43:04.461859041Z Gathering data for ns/openshift-cluster-version...
2026-09-18T17:43:04.621720712Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T17:43:04.732775384Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T17:43:04.821918203Z Gathering data for ns/default...
2026-09-18T17:43:05.151023469Z Gathering data for ns/openshift...
2026-09-18T17:43:05.606426280Z Gathering data for ns/kube-system...
2026-09-18T17:43:06.402111762Z Gathering data for ns/openshift-etcd...
2026-09-18T17:43:06.839350890Z Wrote inspect data to must-gather.
2026-09-18T17:43:06.839388948Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T17:43:06.839388948Z     namespaces "assisted-installer" not found
2026-09-18T17:43:06.941322529Z Waiting on subprocesses to finish execution.
2026-09-18T17:43:06.954318835Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-18T17:43:06.954449401Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-18T17:43:06.957653637Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-18T17:43:06.958714801Z Executing Istio gather script
2026-09-18T17:43:06.965896441Z INFO: Collecting host service logs for crio
2026-09-18T17:43:06.966203454Z INFO: Collecting host service logs for kubelet
2026-09-18T17:43:06.966585986Z INFO: Collecting host service logs for rpm-ostreed
2026-09-18T17:43:06.966956086Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-18T17:43:06.967297315Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-18T17:43:06.967682127Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-18T17:43:06.968046574Z INFO: Collecting host service logs for NetworkManager
2026-09-18T17:43:06.968381526Z INFO: Collecting host service logs for openvswitch
2026-09-18T17:43:06.968732478Z INFO: Collecting host service logs for ovs-configuration
2026-09-18T17:43:06.969086770Z INFO: Collecting host service logs for ovsdb-server
2026-09-18T17:43:06.969420288Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-18T17:43:06.969794121Z INFO: Waiting for worker host service log collection to complete ...
2026-09-18T17:43:06.971651447Z INFO: Gathering HAProxy config files
2026-09-18T17:43:07.190352707Z INFO: Waiting for node performance related collection to complete ...
2026-09-18T17:43:07.429460344Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-18T17:43:07.502090172Z No resources found
2026-09-18T17:43:07.842111815Z INFO: "metallb-operator" not detected. Skipping.
2026-09-18T17:43:07.921250570Z error: the server doesn't have a resource type "performanceprofile"
2026-09-18T17:43:08.022599782Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-18T17:43:08.099370056Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-18T17:43:08.099462514Z INFO: on-disk MachineConfig config collection complete.
2026-09-18T17:43:08.137136071Z Wrote inspect data to must-gather.
2026-09-18T17:43:08.317664387Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-18T17:43:08.346931576Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-18T17:43:08.410100739Z No resources found in openshift-etcd namespace.
2026-09-18T17:43:08.442393145Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-588s7
2026-09-18T17:43:08.442441725Z insights-runtime-extractor-4289r
2026-09-18T17:43:08.442462169Z insights-runtime-extractor-cpk5r
2026-09-18T17:43:08.442481864Z insights-runtime-extractor-tr2tv
2026-09-18T17:43:08.556469507Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-18T17:43:08.706589537Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-18T17:43:08.809549718Z ERROR: No running kube-apiserver pods found
2026-09-18T17:43:08.866054812Z Gathering data for ns/openshift-monitoring...
2026-09-18T17:43:08.935142678Z INFO: 'previous-logs' folder not found on ip-10-0-131-119.ec2.internal, skipping...
2026-09-18T17:43:08.947305219Z Wrote inspect data to must-gather.
2026-09-18T17:43:08.964938767Z INFO: Worker host service log collection to complete.
2026-09-18T17:43:09.158247114Z Wrote inspect data to must-gather.
2026-09-18T17:43:09.252355634Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-18T17:43:09.278621010Z tar: Removing leading `/' from member names
2026-09-18T17:43:09.283779651Z INFO: HAProxy config collection complete.
2026-09-18T17:43:09.317285113Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-18T17:43:09.374073780Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T17:43:09.376190481Z Wrote inspect data to must-gather.
2026-09-18T17:43:09.407180901Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T17:43:09.410171387Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-18T17:43:09.410171387Z See 'oc rsync -h' for help and examples
2026-09-18T17:43:09.411894024Z Wrote inspect data to must-gather.
2026-09-18T17:43:09.492886188Z No resources found
2026-09-18T17:43:09.583325765Z INFO: Getting rules from prometheus-k8s-0
2026-09-18T17:43:09.584625013Z Wrote inspect data to must-gather.
2026-09-18T17:43:09.607285571Z error: the server doesn't have a resource type "machineconfigs"
2026-09-18T17:43:09.676350816Z Wrote inspect data to must-gather.
2026-09-18T17:43:09.694059536Z INFO: 'previous-logs' folder not found on ip-10-0-134-250.ec2.internal, skipping...
2026-09-18T17:43:09.792165592Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-18T17:43:09.879773884Z Wrote inspect data to must-gather.
2026-09-18T17:43:09.981917165Z INFO: 'previous-logs' folder not found on ip-10-0-135-98.ec2.internal, skipping...
2026-09-18T17:43:09.981950847Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-18T17:43:09.981980940Z INFO: Machine Config Daemon termination log collection complete.
2026-09-18T17:43:10.031216961Z Wrote inspect data to must-gather.
2026-09-18T17:43:10.047583778Z INFO: Getting status/config from prometheus-k8s-0
2026-09-18T17:43:10.143296352Z Wrote inspect data to must-gather.
2026-09-18T17:43:10.193738914Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-18T17:43:10.371828284Z Wrote inspect data to must-gather.
2026-09-18T17:43:10.398188705Z Wrote inspect data to must-gather.
2026-09-18T17:43:10.472623725Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-18T17:43:10.614895635Z Wrote inspect data to must-gather.
2026-09-18T17:43:10.700888168Z INFO: INTERCONNECT MODE
2026-09-18T17:43:10.700936113Z INFO: Gathering ovn-kubernetes DBs
2026-09-18T17:43:10.730921586Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-18T17:43:10.770262309Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-18T17:43:10.813112611Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-18T17:43:10.857816318Z Wrote inspect data to must-gather.
2026-09-18T17:43:10.877707270Z INFO: Gathering OVN_Northbound from ovnkube-node-6wsb2...
2026-09-18T17:43:10.877966652Z INFO: Gathering OVN_Northbound from ovnkube-node-mctq4...
2026-09-18T17:43:10.878205545Z INFO: Gathering OVN_Northbound from ovnkube-node-p4j2x...
2026-09-18T17:43:10.878530460Z INFO: Gathering OVN_Southbound from ovnkube-node-6wsb2...
2026-09-18T17:43:10.878970016Z INFO: Gathering OVN_Southbound from ovnkube-node-mctq4...
2026-09-18T17:43:10.882627723Z INFO: Gathering OVN_Southbound from ovnkube-node-p4j2x...
2026-09-18T17:43:11.059078200Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:11.064653564Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-18T17:43:11.222681315Z tar: Removing leading `/' from member names
2026-09-18T17:43:11.226748578Z tar: Removing leading `/' from member names
2026-09-18T17:43:11.226786223Z tar: Removing leading `/' from member names
2026-09-18T17:43:11.254075714Z tar: Removing leading `/' from member names
2026-09-18T17:43:11.270406082Z Wrote inspect data to must-gather.
2026-09-18T17:43:11.282326033Z tar: Removing leading `/' from member names
2026-09-18T17:43:11.300336053Z tar: Removing leading `/' from member names
2026-09-18T17:43:11.385865477Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:11.388556037Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-18T17:43:11.407357966Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-18T17:43:11.516054605Z Wrote inspect data to must-gather.
2026-09-18T17:43:11.644096230Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:11.672947179Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-18T17:43:11.752258023Z INFO: Waiting for network log collection to complete ...
2026-09-18T17:43:11.752374158Z INFO: Waiting for ovnk database copies to complete ...
2026-09-18T17:43:11.752907863Z INFO: Copying ovnk databases complete.
2026-09-18T17:43:11.755003932Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-09-18T17:43:11.757860258Z ovnk_database_store/
2026-09-18T17:43:11.757860258Z ovnk_database_store/ovnkube-node-mctq4_nbdb
2026-09-18T17:43:11.766448837Z ovnk_database_store/ovnkube-node-6wsb2_sbdb
2026-09-18T17:43:11.784643976Z ovnk_database_store/ovnkube-node-6wsb2_nbdb
2026-09-18T17:43:11.791444940Z ovnk_database_store/ovnkube-node-p4j2x_sbdb
2026-09-18T17:43:11.806999693Z ovnk_database_store/ovnkube-node-mctq4_sbdb
2026-09-18T17:43:11.822797065Z ovnk_database_store/ovnkube-node-p4j2x_nbdb
2026-09-18T17:43:11.833438762Z INFO: Network log collection complete.
2026-09-18T17:43:11.846821391Z Wrote inspect data to must-gather.
2026-09-18T17:43:11.892058893Z Gathering data for ns/openshift-network-console...
2026-09-18T17:43:11.894417787Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:11.897344496Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-18T17:43:11.918744926Z INFO: Getting status from alertmanager-main-0
2026-09-18T17:43:12.115834155Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:12.164354061Z Wrote inspect data to must-gather.
2026-09-18T17:43:12.336056232Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:12.339868771Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-18T17:43:12.377168797Z Gathering data for ns/openshift-console-operator...
2026-09-18T17:43:12.436204795Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-18T17:43:12.438688406Z INFO: Fallback to identify the container image from release info
2026-09-18T17:43:12.559293997Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:12.750619398Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:12.753530334Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-18T17:43:12.775679021Z Gathering data for ns/openshift-console...
2026-09-18T17:43:12.923907745Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:13.043988808Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-18T17:43:13.114866635Z Wrote inspect data to /must-gather/istio.
2026-09-18T17:43:13.185331632Z daemonset.apps/perf-node-gather-daemonset created
2026-09-18T17:43:13.335704334Z Waiting for performance profile collector pods to become ready: 1
2026-09-18T17:43:13.335847752Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-18T17:43:13.839722922Z Done executing Istio gather script
2026-09-18T17:43:13.881454380Z Gathering data for ns/openshift-dns-operator...
2026-09-18T17:43:14.203123825Z Gathering data for ns/openshift-dns...
2026-09-18T17:43:14.381743539Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T17:43:14.443821599Z Waiting for performance profile collector pods to become ready: 2
2026-09-18T17:43:14.810795018Z Gathering data for ns/openshift-image-registry...
2026-09-18T17:43:15.291501169Z Gathering data for ns/openshift-ingress-operator...
2026-09-18T17:43:15.570446079Z Waiting for performance profile collector pods to become ready: 3
2026-09-18T17:43:15.674453756Z Gathering data for ns/openshift-ingress...
2026-09-18T17:43:16.013844234Z Gathering data for ns/openshift-ingress-canary...
2026-09-18T17:43:16.483477408Z Gathering data for ns/openshift-insights...
2026-09-18T17:43:16.675293127Z Waiting for performance profile collector pods to become ready: 4
2026-09-18T17:43:17.793697315Z Waiting for performance profile collector pods to become ready: 5
2026-09-18T17:43:18.433722396Z Gathering data for ns/openshift-config...
2026-09-18T17:43:18.753161562Z Gathering data for ns/openshift-config-managed...
2026-09-18T17:43:18.906485919Z Waiting for performance profile collector pods to become ready: 6
2026-09-18T17:43:19.169049238Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-18T17:43:19.386969955Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T17:43:19.431038577Z Gathering data for ns/openshift-kube-apiserver...
2026-09-18T17:43:19.686351086Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-18T17:43:19.974570766Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-18T17:43:20.039438105Z Waiting for performance profile collector pods to become ready: 7
2026-09-18T17:43:20.289361437Z Gathering data for ns/openshift-kube-scheduler...
2026-09-18T17:43:20.562641080Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-18T17:43:20.963317263Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-18T17:43:21.164026399Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-18T17:43:21.393883212Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-18T17:43:21.426821184Z Collecting performance related data for node ip-10-0-131-119.ec2.internal
2026-09-18T17:43:21.434493719Z Collecting performance related data for node ip-10-0-134-250.ec2.internal
2026-09-18T17:43:21.441626878Z Collecting performance related data for node ip-10-0-135-98.ec2.internal
2026-09-18T17:43:21.836557900Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-18T17:43:22.452022439Z Gathering data for ns/openshift-multus...
2026-09-18T17:43:24.067875368Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-18T17:43:24.392981105Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T17:43:25.629549910Z Gathering data for ns/openshift-host-network...
2026-09-18T17:43:26.036357946Z Gathering data for ns/openshift-network-diagnostics...
2026-09-18T17:43:26.902828984Z Gathering data for ns/openshift-network-node-identity...
2026-09-18T17:43:27.338039118Z Gathering data for ns/openshift-network-operator...
2026-09-18T17:43:27.735421118Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-18T17:43:27.987928919Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-18T17:43:28.504956989Z Gathering data for ns/openshift-apiserver-operator...
2026-09-18T17:43:28.828385366Z Gathering data for ns/openshift-apiserver...
2026-09-18T17:43:29.241878467Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-18T17:43:29.398539378Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-18T17:43:29.520454051Z Gathering data for ns/openshift-controller-manager...
2026-09-18T17:43:29.828212287Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-18T17:43:29.849481086Z Collecting kubelet logs for node ip-10-0-131-119.ec2.internal
2026-09-18T17:43:29.851402505Z Collecting kubelet logs for node ip-10-0-134-250.ec2.internal
2026-09-18T17:43:29.853579682Z Collecting kubelet logs for node ip-10-0-135-98.ec2.internal
2026-09-18T17:43:30.193319994Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-qhrzl namespace
2026-09-18T17:43:30.199374301Z INFO: Node performance data collection complete.
2026-09-18T17:43:30.604029166Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-18T17:43:30.884926249Z Gathering data for ns/openshift-service-ca-operator...
2026-09-18T17:43:31.196955349Z Gathering data for ns/openshift-service-ca...
2026-09-18T17:43:31.668099069Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-18T17:43:32.653420672Z Wrote inspect data to must-gather.
2026-09-18T17:43:32.653468464Z error: inspection completed with the errors occurred while gathering data:
2026-09-18T17:43:32.653468464Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-18T17:43:32.781842445Z error: the server doesn't have a resource type "clusters"
2026-09-18T17:43:32.877615639Z Caches written to disk