2026-09-16T13:14:56.923367577Z [disk usage checker] Started 2026-09-16T13:14:56.926727872Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-09-16T13:14:57.116332480Z Gathering data for ns/openshift-cluster-version... 2026-09-16T13:14:57.250088831Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T13:14:57.338972862Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T13:14:57.425962782Z Gathering data for ns/default... 2026-09-16T13:14:57.712223967Z Gathering data for ns/openshift... 2026-09-16T13:14:58.103193744Z Gathering data for ns/kube-system... 2026-09-16T13:14:58.731243258Z Gathering data for ns/openshift-etcd... 2026-09-16T13:14:59.016441685Z Wrote inspect data to must-gather. 2026-09-16T13:14:59.016475303Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T13:14:59.016475303Z namespaces "assisted-installer" not found 2026-09-16T13:14:59.595975872Z Waiting on subprocesses to finish execution. 2026-09-16T13:14:59.602646441Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-16T13:14:59.610021220Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-16T13:14:59.612160962Z Executing Istio gather script 2026-09-16T13:14:59.622281410Z INFO: Gathering HAProxy config files 2026-09-16T13:14:59.628920963Z INFO: Collecting host service logs for crio 2026-09-16T13:14:59.629215546Z INFO: Collecting host service logs for kubelet 2026-09-16T13:14:59.629619750Z INFO: Collecting host service logs for rpm-ostreed 2026-09-16T13:14:59.630002745Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-16T13:14:59.630366944Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-16T13:14:59.630743115Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-16T13:14:59.630743115Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-16T13:14:59.631105336Z INFO: Collecting host service logs for NetworkManager 2026-09-16T13:14:59.631573004Z INFO: Collecting host service logs for openvswitch 2026-09-16T13:14:59.631932671Z INFO: Collecting host service logs for ovs-configuration 2026-09-16T13:14:59.632283384Z INFO: Collecting host service logs for ovsdb-server 2026-09-16T13:14:59.632635094Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-16T13:14:59.633132681Z INFO: Waiting for worker host service log collection to complete ... 2026-09-16T13:14:59.646548676Z INFO: Waiting for node performance related collection to complete ... 2026-09-16T13:15:00.292630205Z INFO: Collecting Insights Archives from insights-operator-5844b856db-jgggh 2026-09-16T13:15:00.292630205Z insights-runtime-extractor-2m9hh 2026-09-16T13:15:00.292630205Z insights-runtime-extractor-kf9s5 2026-09-16T13:15:00.292630205Z insights-runtime-extractor-tjwqw 2026-09-16T13:15:00.379555320Z No resources found in openshift-etcd namespace. 2026-09-16T13:15:00.666273172Z No resources found 2026-09-16T13:15:00.770287492Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-16T13:15:00.839265249Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-16T13:15:00.857842485Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-16T13:15:00.978803154Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-16T13:15:01.002851790Z Wrote inspect data to must-gather. 2026-09-16T13:15:01.053815633Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-16T13:15:01.053815633Z See 'oc rsync -h' for help and examples 2026-09-16T13:15:01.059617155Z INFO: "metallb-operator" not detected. Skipping. 2026-09-16T13:15:01.203130472Z INFO: Worker host service log collection to complete. 2026-09-16T13:15:01.213579631Z error: the server doesn't have a resource type "performanceprofile" 2026-09-16T13:15:01.218139367Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-16T13:15:01.218166935Z INFO: on-disk MachineConfig config collection complete. 2026-09-16T13:15:01.322126277Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-16T13:15:01.504463630Z Wrote inspect data to must-gather. 2026-09-16T13:15:01.580124928Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-16T13:15:01.659479182Z Wrote inspect data to must-gather. 2026-09-16T13:15:01.726013538Z No resources found 2026-09-16T13:15:01.875873634Z Wrote inspect data to must-gather. 2026-09-16T13:15:01.943290364Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-09-16T13:15:02.013877428Z Wrote inspect data to must-gather. 2026-09-16T13:15:02.038437150Z ERROR: No running kube-apiserver pods found 2026-09-16T13:15:02.109363351Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T13:15:02.178475213Z Gathering data for ns/openshift-monitoring... 2026-09-16T13:15:02.213513361Z Wrote inspect data to must-gather. 2026-09-16T13:15:02.222076106Z tar: Removing leading `/' from member names 2026-09-16T13:15:02.225260864Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-16T13:15:02.228992115Z INFO: HAProxy config collection complete. 2026-09-16T13:15:02.324960296Z INFO: 'previous-logs' folder not found on ip-10-0-131-158.ec2.internal, skipping... 2026-09-16T13:15:02.419471064Z Wrote inspect data to must-gather. 2026-09-16T13:15:02.547671566Z Wrote inspect data to must-gather. 2026-09-16T13:15:02.550454794Z INFO: Getting rules from prometheus-k8s-0 2026-09-16T13:15:02.646660839Z INFO: 'previous-logs' folder not found on ip-10-0-133-218.ec2.internal, skipping... 2026-09-16T13:15:02.657692974Z error: the server doesn't have a resource type "machineconfigs" 2026-09-16T13:15:02.853919061Z Wrote inspect data to must-gather. 2026-09-16T13:15:02.855467030Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-16T13:15:02.953021527Z INFO: 'previous-logs' folder not found on ip-10-0-143-99.ec2.internal, skipping... 2026-09-16T13:15:02.953126802Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-16T13:15:02.953185064Z INFO: Machine Config Daemon termination log collection complete. 2026-09-16T13:15:03.034448145Z INFO: Getting status/config from prometheus-k8s-0 2026-09-16T13:15:03.116432778Z Wrote inspect data to must-gather. 2026-09-16T13:15:03.140928332Z Wrote inspect data to must-gather. 2026-09-16T13:15:03.307254916Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-16T13:15:03.499205458Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-16T13:15:03.532099182Z INFO: INTERCONNECT MODE 2026-09-16T13:15:03.532190365Z INFO: Gathering ovn-kubernetes DBs 2026-09-16T13:15:03.549133669Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-16T13:15:03.621379613Z Wrote inspect data to must-gather. 2026-09-16T13:15:03.676013135Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T13:15:03.728147476Z INFO: Gathering OVN_Northbound from ovnkube-node-894fk... 2026-09-16T13:15:03.728362876Z INFO: Gathering OVN_Northbound from ovnkube-node-q8psw... 2026-09-16T13:15:03.728584747Z INFO: Gathering OVN_Northbound from ovnkube-node-rwjbf... 2026-09-16T13:15:03.729005544Z INFO: Gathering OVN_Southbound from ovnkube-node-894fk... 2026-09-16T13:15:03.729449605Z INFO: Gathering OVN_Southbound from ovnkube-node-q8psw... 2026-09-16T13:15:03.729738918Z INFO: Gathering OVN_Southbound from ovnkube-node-rwjbf... 2026-09-16T13:15:03.753362631Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-16T13:15:04.044018262Z tar: Removing leading `/' from member names 2026-09-16T13:15:04.048334587Z Wrote inspect data to must-gather. 2026-09-16T13:15:04.098366893Z tar: Removing leading `/' from member names 2026-09-16T13:15:04.125556004Z tar: Removing leading `/' from member names 2026-09-16T13:15:04.134329844Z tar: Removing leading `/' from member names 2026-09-16T13:15:04.142047390Z tar: Removing leading `/' from member names 2026-09-16T13:15:04.149015890Z tar: Removing leading `/' from member names 2026-09-16T13:15:04.168241930Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-16T13:15:04.315070503Z Wrote inspect data to must-gather. 2026-09-16T13:15:04.428325602Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-16T13:15:04.479599004Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T13:15:04.563084554Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-16T13:15:04.694135848Z Gathering data for ns/openshift-network-console... 2026-09-16T13:15:04.698196450Z Wrote inspect data to must-gather. 2026-09-16T13:15:04.718467139Z INFO: Waiting for network log collection to complete ... 2026-09-16T13:15:04.718515204Z INFO: Waiting for ovnk database copies to complete ... 2026-09-16T13:15:04.718931800Z INFO: Copying ovnk databases complete. 2026-09-16T13:15:04.721038363Z 3.7M must-gather/network_logs/ovnk_database_store 2026-09-16T13:15:04.723852454Z ovnk_database_store/ 2026-09-16T13:15:04.723852454Z ovnk_database_store/ovnkube-node-q8psw_sbdb 2026-09-16T13:15:04.757179230Z ovnk_database_store/ovnkube-node-rwjbf_nbdb 2026-09-16T13:15:04.767124512Z ovnk_database_store/ovnkube-node-894fk_sbdb 2026-09-16T13:15:04.791635001Z ovnk_database_store/ovnkube-node-rwjbf_sbdb 2026-09-16T13:15:04.793951780Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:04.816012791Z ovnk_database_store/ovnkube-node-q8psw_nbdb 2026-09-16T13:15:04.826624413Z ovnk_database_store/ovnkube-node-894fk_nbdb 2026-09-16T13:15:04.840321740Z INFO: Network log collection complete. 2026-09-16T13:15:04.897011966Z INFO: Getting status from alertmanager-main-0 2026-09-16T13:15:05.034682688Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:05.037898598Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-16T13:15:05.054100574Z Wrote inspect data to must-gather. 2026-09-16T13:15:05.214055272Z Gathering data for ns/openshift-console-operator... 2026-09-16T13:15:05.257143152Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:05.377082291Z Wrote inspect data to must-gather. 2026-09-16T13:15:05.477141890Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:05.480215896Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-16T13:15:05.624274722Z Gathering data for ns/openshift-console... 2026-09-16T13:15:05.680974340Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:05.704640504Z Wrote inspect data to must-gather. 2026-09-16T13:15:05.898693725Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:05.902010323Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-16T13:15:05.995837482Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-16T13:15:05.998987692Z INFO: Fallback to identify the container image from release info 2026-09-16T13:15:06.133263014Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:06.187369659Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-16T13:15:06.332066916Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:06.334248275Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-16T13:15:06.512562694Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:06.703690546Z Gathering data for ns/openshift-dns-operator... 2026-09-16T13:15:06.715498177Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:06.718720406Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-16T13:15:06.900105729Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:06.957375201Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-09-16T13:15:06.968579903Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-16T13:15:07.018224611Z Gathering data for ns/openshift-dns... 2026-09-16T13:15:07.107128777Z daemonset.apps/perf-node-gather-daemonset created 2026-09-16T13:15:07.113307417Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:07.116046962Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-16T13:15:07.236828940Z Waiting for performance profile collector pods to become ready: 1 2026-09-16T13:15:07.302530428Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:07.499085663Z Wrote inspect data to /must-gather/istio. 2026-09-16T13:15:07.670010435Z Gathering data for ns/openshift-image-registry... 2026-09-16T13:15:08.182228308Z Gathering data for ns/openshift-ingress-operator... 2026-09-16T13:15:08.233273450Z Done executing Istio gather script 2026-09-16T13:15:08.352662714Z Waiting for performance profile collector pods to become ready: 2 2026-09-16T13:15:08.522815642Z Gathering data for ns/openshift-ingress... 2026-09-16T13:15:08.824484326Z Gathering data for ns/openshift-ingress-canary... 2026-09-16T13:15:09.174941280Z Gathering data for ns/openshift-insights... 2026-09-16T13:15:09.478240689Z Waiting for performance profile collector pods to become ready: 3 2026-09-16T13:15:10.594102810Z Waiting for performance profile collector pods to become ready: 4 2026-09-16T13:15:11.518578703Z Gathering data for ns/kserve... 2026-09-16T13:15:11.713415829Z Waiting for performance profile collector pods to become ready: 5 2026-09-16T13:15:11.962450062Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-09-16T13:15:12.830095750Z Waiting for performance profile collector pods to become ready: 6 2026-09-16T13:15:13.092771951Z Gathering data for ns/openshift-config... 2026-09-16T13:15:13.334536691Z Gathering data for ns/openshift-config-managed... 2026-09-16T13:15:13.736580086Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-16T13:15:13.956019787Z Waiting for performance profile collector pods to become ready: 7 2026-09-16T13:15:13.997784212Z Gathering data for ns/openshift-kube-apiserver... 2026-09-16T13:15:14.292553830Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-16T13:15:14.620106908Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-16T13:15:14.929823214Z Gathering data for ns/openshift-kube-scheduler... 2026-09-16T13:15:15.081558946Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-16T13:15:15.205481541Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-16T13:15:15.365027642Z Collecting performance related data for node ip-10-0-131-158.ec2.internal 2026-09-16T13:15:15.372226025Z Collecting performance related data for node ip-10-0-133-218.ec2.internal 2026-09-16T13:15:15.383562911Z Collecting performance related data for node ip-10-0-143-99.ec2.internal 2026-09-16T13:15:15.558289876Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-16T13:15:15.928938397Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-16T13:15:16.310508410Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-16T13:15:16.944067910Z Gathering data for ns/openshift-multus... 2026-09-16T13:15:16.967398221Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-09-16T13:15:18.491617346Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-16T13:15:19.649274463Z Gathering data for ns/openshift-host-network... 2026-09-16T13:15:19.981563690Z Gathering data for ns/openshift-network-diagnostics... 2026-09-16T13:15:20.501990250Z Gathering data for ns/openshift-network-node-identity... 2026-09-16T13:15:20.931609897Z Gathering data for ns/openshift-network-operator... 2026-09-16T13:15:21.279540621Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-16T13:15:21.570447956Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-16T13:15:21.973960536Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T13:15:22.044095046Z Gathering data for ns/openshift-apiserver-operator... 2026-09-16T13:15:22.376506165Z Gathering data for ns/openshift-apiserver... 2026-09-16T13:15:22.742683602Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-16T13:15:23.013264444Z Gathering data for ns/openshift-controller-manager... 2026-09-16T13:15:23.314097370Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-16T13:15:23.344624126Z Collecting kubelet logs for node ip-10-0-131-158.ec2.internal 2026-09-16T13:15:23.346472569Z Collecting kubelet logs for node ip-10-0-133-218.ec2.internal 2026-09-16T13:15:23.348318282Z Collecting kubelet logs for node ip-10-0-143-99.ec2.internal 2026-09-16T13:15:23.745006309Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-l458r namespace 2026-09-16T13:15:23.749339267Z INFO: Node performance data collection complete. 2026-09-16T13:15:24.022289910Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-16T13:15:24.338531641Z Gathering data for ns/openshift-service-ca-operator... 2026-09-16T13:15:24.637782070Z Gathering data for ns/openshift-service-ca... 2026-09-16T13:15:24.994177682Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-16T13:15:25.838036732Z Wrote inspect data to must-gather. 2026-09-16T13:15:25.838079156Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T13:15:25.838079156Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-16T13:15:25.967077312Z error: the server doesn't have a resource type "clusters" 2026-09-16T13:15:26.114571718Z Caches written to disk