2026-09-18T21:58:49.707303910Z [disk usage checker] Started 2026-09-18T21:58:49.710894467Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-18T21:58:49.919267819Z Gathering data for ns/openshift-cluster-version... 2026-09-18T21:58:50.098916968Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-18T21:58:50.218144103Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T21:58:50.313796860Z Gathering data for ns/default... 2026-09-18T21:58:50.649730281Z Gathering data for ns/openshift... 2026-09-18T21:58:51.192618322Z Gathering data for ns/kube-system... 2026-09-18T21:58:52.068501731Z Gathering data for ns/openshift-etcd... 2026-09-18T21:58:52.486976926Z Waiting on subprocesses to finish execution. 2026-09-18T21:58:52.495510300Z INFO: Gathering HAProxy config files 2026-09-18T21:58:52.501329894Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-18T21:58:52.502482268Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-18T21:58:52.503611359Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-18T21:58:52.520453292Z Executing Istio gather script 2026-09-18T21:58:52.529366923Z INFO: Collecting host service logs for crio 2026-09-18T21:58:52.529767631Z INFO: Collecting host service logs for kubelet 2026-09-18T21:58:52.530126257Z INFO: Collecting host service logs for rpm-ostreed 2026-09-18T21:58:52.530476878Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-18T21:58:52.530818233Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-18T21:58:52.531142391Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-18T21:58:52.531483803Z INFO: Collecting host service logs for NetworkManager 2026-09-18T21:58:52.531800592Z INFO: Collecting host service logs for openvswitch 2026-09-18T21:58:52.532139447Z INFO: Collecting host service logs for ovs-configuration 2026-09-18T21:58:52.532492307Z INFO: Collecting host service logs for ovsdb-server 2026-09-18T21:58:52.532828735Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-18T21:58:52.533222811Z INFO: Waiting for worker host service log collection to complete ... 2026-09-18T21:58:52.559289142Z Wrote inspect data to must-gather. 2026-09-18T21:58:52.559327234Z error: inspection completed with the errors occurred while gathering data: 2026-09-18T21:58:52.559327234Z namespaces "assisted-installer" not found 2026-09-18T21:58:52.573114121Z INFO: Waiting for node performance related collection to complete ... 2026-09-18T21:58:53.218253862Z error: the server doesn't have a resource type "performanceprofile" 2026-09-18T21:58:53.336829193Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-18T21:58:53.368461711Z INFO: "metallb-operator" not detected. Skipping. 2026-09-18T21:58:53.511062507Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-18T21:58:53.530242412Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-18T21:58:53.642160919Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-18T21:58:53.655450754Z No resources found in openshift-etcd namespace. 2026-09-18T21:58:53.764978574Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-18T21:58:53.974327287Z ERROR: No running kube-apiserver pods found 2026-09-18T21:58:54.002701303Z Wrote inspect data to must-gather. 2026-09-18T21:58:54.063570011Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-bcrfg 2026-09-18T21:58:54.063614387Z insights-runtime-extractor-8ppz7 2026-09-18T21:58:54.063634422Z insights-runtime-extractor-nrzrx 2026-09-18T21:58:54.063653453Z insights-runtime-extractor-pssm9 2026-09-18T21:58:54.137828978Z No resources found 2026-09-18T21:58:54.522525124Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-18T21:58:54.529096795Z Wrote inspect data to must-gather. 2026-09-18T21:58:54.573822357Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-18T21:58:54.573905088Z INFO: on-disk MachineConfig config collection complete. 2026-09-18T21:58:54.730287031Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-18T21:58:54.730287031Z See 'oc rsync -h' for help and examples 2026-09-18T21:58:54.780953159Z INFO: Worker host service log collection to complete. 2026-09-18T21:58:54.883873705Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-18T21:58:54.883911541Z No resources found 2026-09-18T21:58:54.916600488Z error: the server doesn't have a resource type "machineconfigs" 2026-09-18T21:58:55.000653592Z tar: Removing leading `/' from member names 2026-09-18T21:58:55.005536658Z INFO: HAProxy config collection complete. 2026-09-18T21:58:55.039808610Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T21:58:55.166730397Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.217065857Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.275473804Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.300048793Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.301815201Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-18T21:58:55.309302620Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.310208126Z INFO: 'previous-logs' folder not found on ip-10-0-128-157.ec2.internal, skipping... 2026-09-18T21:58:55.466816388Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-18T21:58:55.483593266Z INFO: Getting rules from prometheus-k8s-0 2026-09-18T21:58:55.613898669Z INFO: 'previous-logs' folder not found on ip-10-0-131-121.ec2.internal, skipping... 2026-09-18T21:58:55.616807535Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.730699166Z Gathering data for ns/openshift-monitoring... 2026-09-18T21:58:55.755221497Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.765375532Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-18T21:58:55.888812410Z Wrote inspect data to must-gather. 2026-09-18T21:58:55.936258382Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-18T21:58:55.965997856Z INFO: 'previous-logs' folder not found on ip-10-0-142-88.ec2.internal, skipping... 2026-09-18T21:58:55.966098209Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-18T21:58:55.966145406Z INFO: Machine Config Daemon termination log collection complete. 2026-09-18T21:58:55.990570620Z INFO: Getting status/config from prometheus-k8s-0 2026-09-18T21:58:56.153335779Z Wrote inspect data to must-gather. 2026-09-18T21:58:56.161337684Z Wrote inspect data to must-gather. 2026-09-18T21:58:56.369165346Z Wrote inspect data to must-gather. 2026-09-18T21:58:56.433475617Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-18T21:58:56.445325267Z INFO: INTERCONNECT MODE 2026-09-18T21:58:56.445369809Z INFO: Gathering ovn-kubernetes DBs 2026-09-18T21:58:56.600657762Z Wrote inspect data to must-gather. 2026-09-18T21:58:56.615669755Z INFO: Gathering OVN_Northbound from ovnkube-node-7vxbj... 2026-09-18T21:58:56.615864585Z INFO: Gathering OVN_Northbound from ovnkube-node-r2gbj... 2026-09-18T21:58:56.616071536Z INFO: Gathering OVN_Northbound from ovnkube-node-xfzhg... 2026-09-18T21:58:56.616342714Z INFO: Gathering OVN_Southbound from ovnkube-node-7vxbj... 2026-09-18T21:58:56.616603610Z INFO: Gathering OVN_Southbound from ovnkube-node-r2gbj... 2026-09-18T21:58:56.616862863Z INFO: Gathering OVN_Southbound from ovnkube-node-xfzhg... 2026-09-18T21:58:56.683179281Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-18T21:58:56.955881634Z Wrote inspect data to must-gather. 2026-09-18T21:58:57.003630591Z tar: Removing leading `/' from member names 2026-09-18T21:58:57.025358701Z tar: Removing leading `/' from member names 2026-09-18T21:58:57.045794365Z tar: Removing leading `/' from member names 2026-09-18T21:58:57.085534056Z tar: Removing leading `/' from member names 2026-09-18T21:58:57.186825233Z tar: Removing leading `/' from member names 2026-09-18T21:58:57.198739737Z tar: Removing leading `/' from member names 2026-09-18T21:58:57.199795796Z Wrote inspect data to must-gather. 2026-09-18T21:58:57.231110727Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-18T21:58:57.355472838Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-18T21:58:57.438105084Z Wrote inspect data to must-gather. 2026-09-18T21:58:57.442820916Z INFO: Waiting for network log collection to complete ... 2026-09-18T21:58:57.442843727Z INFO: Waiting for ovnk database copies to complete ... 2026-09-18T21:58:57.443207518Z INFO: Copying ovnk databases complete. 2026-09-18T21:58:57.444687458Z 2.8M must-gather/network_logs/ovnk_database_store 2026-09-18T21:58:57.446727944Z ovnk_database_store/ 2026-09-18T21:58:57.446727944Z ovnk_database_store/ovnkube-node-7vxbj_nbdb 2026-09-18T21:58:57.453025311Z ovnk_database_store/ovnkube-node-r2gbj_sbdb 2026-09-18T21:58:57.469358316Z ovnk_database_store/ovnkube-node-xfzhg_sbdb 2026-09-18T21:58:57.486567336Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-18T21:58:57.490030302Z ovnk_database_store/ovnkube-node-7vxbj_sbdb 2026-09-18T21:58:57.505096019Z ovnk_database_store/ovnkube-node-xfzhg_nbdb 2026-09-18T21:58:57.512168365Z ovnk_database_store/ovnkube-node-r2gbj_nbdb 2026-09-18T21:58:57.524278533Z INFO: Network log collection complete. 2026-09-18T21:58:57.544049353Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:57.649713713Z Wrote inspect data to must-gather. 2026-09-18T21:58:57.755702136Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:57.758971822Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-18T21:58:57.905944325Z INFO: Getting status from alertmanager-main-0 2026-09-18T21:58:57.916434780Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-18T21:58:57.950919437Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-18T21:58:57.952712037Z INFO: Fallback to identify the container image from release info 2026-09-18T21:58:57.980564640Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:58.205439445Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:58.208057421Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-18T21:58:58.395857063Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:58.442664358Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea 2026-09-18T21:58:58.497729474Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-18T21:58:58.581221158Z daemonset.apps/perf-node-gather-daemonset created 2026-09-18T21:58:58.604903225Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:58.608032487Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-18T21:58:58.707136048Z Gathering data for ns/openshift-network-console... 2026-09-18T21:58:58.740149757Z Waiting for performance profile collector pods to become ready: 1 2026-09-18T21:58:58.858738094Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:59.062490921Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:59.065288707Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-18T21:58:59.256466718Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:59.453997747Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:59.456313707Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-18T21:58:59.457833458Z Gathering data for ns/openshift-console-operator... 2026-09-18T21:58:59.622488758Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:59.808887717Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:58:59.811896113Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-18T21:58:59.889064745Z Waiting for performance profile collector pods to become ready: 2 2026-09-18T21:58:59.908182175Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-18T21:58:59.924136447Z Gathering data for ns/openshift-console... 2026-09-18T21:58:59.983283751Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:59:00.178235399Z Wrote inspect data to /must-gather/istio. 2026-09-18T21:59:00.498186232Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-18T21:59:00.858987732Z Done executing Istio gather script 2026-09-18T21:59:01.016058924Z Waiting for performance profile collector pods to become ready: 3 2026-09-18T21:59:01.054324845Z Gathering data for ns/openshift-dns-operator... 2026-09-18T21:59:01.385652130Z Gathering data for ns/openshift-dns... 2026-09-18T21:59:02.151451997Z Waiting for performance profile collector pods to become ready: 4 2026-09-18T21:59:02.180014513Z Gathering data for ns/openshift-image-registry... 2026-09-18T21:59:02.768626814Z Gathering data for ns/openshift-ingress-operator... 2026-09-18T21:59:03.144044615Z Gathering data for ns/openshift-ingress... 2026-09-18T21:59:03.279823894Z Waiting for performance profile collector pods to become ready: 5 2026-09-18T21:59:03.537975652Z Gathering data for ns/openshift-ingress-canary... 2026-09-18T21:59:03.973317692Z Gathering data for ns/openshift-insights... 2026-09-18T21:59:04.409224551Z Waiting for performance profile collector pods to become ready: 6 2026-09-18T21:59:04.913258518Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-18T21:59:05.523328689Z Waiting for performance profile collector pods to become ready: 7 2026-09-18T21:59:06.622838014Z Gathering data for ns/kserve... 2026-09-18T21:59:06.646173227Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-18T21:59:06.898671217Z Collecting performance related data for node ip-10-0-131-121.ec2.internal 2026-09-18T21:59:06.905171605Z Collecting performance related data for node ip-10-0-142-88.ec2.internal 2026-09-18T21:59:06.912088666Z Collecting performance related data for node ip-10-0-128-157.ec2.internal 2026-09-18T21:59:08.268035112Z Gathering data for ns/openshift-config... 2026-09-18T21:59:08.637936822Z Gathering data for ns/openshift-config-managed... 2026-09-18T21:59:09.121397707Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-18T21:59:09.493629405Z Gathering data for ns/openshift-kube-apiserver... 2026-09-18T21:59:09.896273274Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-18T21:59:09.922216576Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-18T21:59:10.273709369Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-18T21:59:10.625528422Z Gathering data for ns/openshift-kube-scheduler... 2026-09-18T21:59:10.931132653Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-18T21:59:11.314307958Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-18T21:59:11.715388891Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-18T21:59:12.080488346Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-18T21:59:12.627594314Z Gathering data for ns/openshift-multus... 2026-09-18T21:59:14.163519815Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-18T21:59:14.928049739Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-18T21:59:15.211657213Z Collecting kubelet logs for node ip-10-0-131-121.ec2.internal 2026-09-18T21:59:15.213345897Z Collecting kubelet logs for node ip-10-0-142-88.ec2.internal 2026-09-18T21:59:15.215473465Z Collecting kubelet logs for node ip-10-0-128-157.ec2.internal 2026-09-18T21:59:15.579847247Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-sskcx namespace 2026-09-18T21:59:15.588059301Z INFO: Node performance data collection complete. 2026-09-18T21:59:15.726339587Z Gathering data for ns/openshift-host-network... 2026-09-18T21:59:16.155014573Z Gathering data for ns/openshift-network-diagnostics... 2026-09-18T21:59:16.815578532Z Gathering data for ns/openshift-network-node-identity... 2026-09-18T21:59:17.349221348Z Gathering data for ns/openshift-network-operator... 2026-09-18T21:59:17.798643917Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-18T21:59:18.210091951Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-18T21:59:18.749855374Z Gathering data for ns/openshift-apiserver-operator... 2026-09-18T21:59:19.115251299Z Gathering data for ns/openshift-apiserver... 2026-09-18T21:59:19.583534902Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-18T21:59:19.933908565Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-18T21:59:19.985845713Z Gathering data for ns/openshift-controller-manager... 2026-09-18T21:59:20.355167206Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-18T21:59:21.164307338Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-18T21:59:21.527816836Z Gathering data for ns/openshift-service-ca-operator... 2026-09-18T21:59:21.946338001Z Gathering data for ns/openshift-service-ca... 2026-09-18T21:59:22.351810484Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-18T21:59:23.302436520Z Wrote inspect data to must-gather. 2026-09-18T21:59:23.302476298Z error: inspection completed with the errors occurred while gathering data: 2026-09-18T21:59:23.302476298Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-18T21:59:23.429072278Z error: the server doesn't have a resource type "clusters" 2026-09-18T21:59:23.535116437Z Caches written to disk