2026-07-13T19:16:26.192987482Z [disk usage checker] Started 2026-07-13T19:16:26.196762728Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-07-13T19:16:26.401268219Z Gathering data for ns/openshift-cluster-version... 2026-07-13T19:16:26.544844700Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-13T19:16:26.633755179Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-13T19:16:26.719535965Z Gathering data for ns/default... 2026-07-13T19:16:27.028345398Z Gathering data for ns/openshift... 2026-07-13T19:16:27.473836929Z Gathering data for ns/kube-system... 2026-07-13T19:16:28.177487263Z Gathering data for ns/openshift-etcd... 2026-07-13T19:16:28.519924492Z Wrote inspect data to must-gather. 2026-07-13T19:16:28.519956519Z error: inspection completed with the errors occurred while gathering data: 2026-07-13T19:16:28.519956519Z namespaces "assisted-installer" not found 2026-07-13T19:16:28.948136003Z Waiting on subprocesses to finish execution. 2026-07-13T19:16:28.953050278Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-13T19:16:28.959478685Z INFO: Gathering HAProxy config files 2026-07-13T19:16:28.962304590Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-13T19:16:28.965415353Z Executing Istio gather script 2026-07-13T19:16:28.968607134Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-13T19:16:28.985594183Z INFO: Collecting host service logs for crio 2026-07-13T19:16:28.985997719Z INFO: Collecting host service logs for kubelet 2026-07-13T19:16:28.986340786Z INFO: Collecting host service logs for rpm-ostreed 2026-07-13T19:16:28.986686058Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-13T19:16:28.987025111Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-13T19:16:28.987331543Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-13T19:16:28.988000262Z INFO: Collecting host service logs for NetworkManager 2026-07-13T19:16:28.988291203Z INFO: Collecting host service logs for openvswitch 2026-07-13T19:16:28.988602108Z INFO: Collecting host service logs for ovs-configuration 2026-07-13T19:16:28.988892370Z INFO: Collecting host service logs for ovsdb-server 2026-07-13T19:16:28.989168984Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-13T19:16:28.989509178Z INFO: Waiting for worker host service log collection to complete ... 2026-07-13T19:16:29.267714887Z INFO: Waiting for node performance related collection to complete ... 2026-07-13T19:16:29.390674345Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-13T19:16:29.470663666Z No resources found in openshift-etcd namespace. 2026-07-13T19:16:29.573874321Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-13T19:16:29.573956118Z INFO: on-disk MachineConfig config collection complete. 2026-07-13T19:16:29.759461242Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-13T19:16:29.852218723Z No resources found 2026-07-13T19:16:30.013440141Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-kvnjs 2026-07-13T19:16:30.013484520Z insights-runtime-extractor-6dqk8 2026-07-13T19:16:30.013503604Z insights-runtime-extractor-bkttc 2026-07-13T19:16:30.013522963Z insights-runtime-extractor-rd8zf 2026-07-13T19:16:30.112705990Z ERROR: No running kube-apiserver pods found 2026-07-13T19:16:30.136019231Z INFO: "metallb-operator" not detected. Skipping. 2026-07-13T19:16:30.214162750Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-13T19:16:30.380634990Z error: the server doesn't have a resource type "performanceprofile" 2026-07-13T19:16:30.399781826Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-13T19:16:30.504892815Z INFO: Worker host service log collection to complete. 2026-07-13T19:16:30.534732480Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-13T19:16:30.534732480Z See 'oc rsync -h' for help and examples 2026-07-13T19:16:30.620005142Z Wrote inspect data to must-gather. 2026-07-13T19:16:30.965174284Z INFO: 'previous-logs' folder not found on ip-10-0-134-215.ec2.internal, skipping... 2026-07-13T19:16:31.202785269Z Wrote inspect data to must-gather. 2026-07-13T19:16:31.258039466Z No resources found 2026-07-13T19:16:31.297299681Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-07-13T19:16:31.411020515Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-13T19:16:31.423297108Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-13T19:16:31.444330998Z Wrote inspect data to must-gather. 2026-07-13T19:16:31.462213753Z Wrote inspect data to must-gather. 2026-07-13T19:16:31.464195998Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-13T19:16:31.510799868Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-13T19:16:31.537690218Z Wrote inspect data to must-gather. 2026-07-13T19:16:31.588946388Z Wrote inspect data to must-gather. 2026-07-13T19:16:31.614732988Z Wrote inspect data to must-gather. 2026-07-13T19:16:31.776128444Z INFO: 'previous-logs' folder not found on ip-10-0-134-48.ec2.internal, skipping... 2026-07-13T19:16:31.816416678Z tar: Removing leading `/' from member names 2026-07-13T19:16:31.824111961Z INFO: HAProxy config collection complete. 2026-07-13T19:16:31.826405491Z error: the server doesn't have a resource type "machineconfigs" 2026-07-13T19:16:31.879208630Z Gathering data for ns/openshift-monitoring... 2026-07-13T19:16:31.905918947Z Wrote inspect data to must-gather. 2026-07-13T19:16:32.048209695Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-13T19:16:32.090721332Z INFO: Getting rules from prometheus-k8s-0 2026-07-13T19:16:32.142050752Z INFO: 'previous-logs' folder not found on ip-10-0-138-93.ec2.internal, skipping... 2026-07-13T19:16:32.142151387Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-13T19:16:32.142202434Z INFO: Machine Config Daemon termination log collection complete. 2026-07-13T19:16:32.224334019Z Wrote inspect data to must-gather. 2026-07-13T19:16:32.319708532Z Wrote inspect data to must-gather. 2026-07-13T19:16:32.492386702Z Wrote inspect data to must-gather. 2026-07-13T19:16:32.511896449Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-13T19:16:32.567378325Z INFO: Getting status/config from prometheus-k8s-0 2026-07-13T19:16:32.601360618Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-13T19:16:32.759759602Z Wrote inspect data to must-gather. 2026-07-13T19:16:32.831650195Z INFO: INTERCONNECT MODE 2026-07-13T19:16:32.831680429Z INFO: Gathering ovn-kubernetes DBs 2026-07-13T19:16:33.001973574Z Wrote inspect data to must-gather. 2026-07-13T19:16:33.003358321Z INFO: Gathering OVN_Northbound from ovnkube-node-frplb... 2026-07-13T19:16:33.003622500Z INFO: Gathering OVN_Northbound from ovnkube-node-n977w... 2026-07-13T19:16:33.003887447Z INFO: Gathering OVN_Northbound from ovnkube-node-xshqz... 2026-07-13T19:16:33.004210795Z INFO: Gathering OVN_Southbound from ovnkube-node-frplb... 2026-07-13T19:16:33.004496296Z INFO: Gathering OVN_Southbound from ovnkube-node-n977w... 2026-07-13T19:16:33.004861851Z INFO: Gathering OVN_Southbound from ovnkube-node-xshqz... 2026-07-13T19:16:33.033353059Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-13T19:16:33.355915552Z tar: Removing leading `/' from member names 2026-07-13T19:16:33.362728177Z tar: Removing leading `/' from member names 2026-07-13T19:16:33.387836576Z tar: Removing leading `/' from member names 2026-07-13T19:16:33.402888224Z tar: Removing leading `/' from member names 2026-07-13T19:16:33.408753640Z tar: Removing leading `/' from member names 2026-07-13T19:16:33.420287013Z tar: Removing leading `/' from member names 2026-07-13T19:16:33.446278778Z Wrote inspect data to must-gather. 2026-07-13T19:16:33.461443713Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-13T19:16:33.533676585Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-13T19:16:33.631799180Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-13T19:16:33.699276440Z Wrote inspect data to must-gather. 2026-07-13T19:16:33.757287917Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-13T19:16:33.882275156Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:33.956412817Z INFO: Waiting for network log collection to complete ... 2026-07-13T19:16:33.956456219Z INFO: Waiting for ovnk database copies to complete ... 2026-07-13T19:16:33.957086151Z INFO: Copying ovnk databases complete. 2026-07-13T19:16:33.958813865Z 3.3M must-gather/network_logs/ovnk_database_store 2026-07-13T19:16:33.961899672Z Wrote inspect data to must-gather. 2026-07-13T19:16:33.962794271Z ovnk_database_store/ 2026-07-13T19:16:33.962794271Z ovnk_database_store/ovnkube-node-xshqz_sbdb 2026-07-13T19:16:33.979836369Z ovnk_database_store/ovnkube-node-xshqz_nbdb 2026-07-13T19:16:33.987853888Z ovnk_database_store/ovnkube-node-n977w_sbdb 2026-07-13T19:16:34.010243888Z ovnk_database_store/ovnkube-node-frplb_nbdb 2026-07-13T19:16:34.020795721Z ovnk_database_store/ovnkube-node-frplb_sbdb 2026-07-13T19:16:34.043974437Z ovnk_database_store/ovnkube-node-n977w_nbdb 2026-07-13T19:16:34.053804754Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-13T19:16:34.057054055Z INFO: Network log collection complete. 2026-07-13T19:16:34.129514741Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:34.132484971Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-13T19:16:34.231913212Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-13T19:16:34.315939285Z Wrote inspect data to must-gather. 2026-07-13T19:16:34.367592264Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:34.449273651Z Gathering data for ns/openshift-network-console... 2026-07-13T19:16:34.509782313Z INFO: Getting status from alertmanager-main-0 2026-07-13T19:16:34.619897847Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:34.622393087Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-13T19:16:34.629933377Z Wrote inspect data to must-gather. 2026-07-13T19:16:34.835431723Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:34.922738815Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-13T19:16:34.925545400Z INFO: Fallback to identify the container image from release info 2026-07-13T19:16:34.967869795Z Gathering data for ns/openshift-console-operator... 2026-07-13T19:16:35.061139101Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:35.065357686Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-13T19:16:35.282542315Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:35.369780593Z Gathering data for ns/openshift-console... 2026-07-13T19:16:35.494123822Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:35.497145498Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io 2026-07-13T19:16:35.653025115Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:35.873777093Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:35.876271487Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io 2026-07-13T19:16:35.899790869Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d 2026-07-13T19:16:35.936981785Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-13T19:16:36.048819325Z daemonset.apps/perf-node-gather-daemonset created 2026-07-13T19:16:36.082657899Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:36.201806274Z Waiting for performance profile collector pods to become ready: 1 2026-07-13T19:16:36.289832059Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:36.293456179Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io 2026-07-13T19:16:36.463766754Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-07-13T19:16:36.489102162Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:36.609935869Z Gathering data for ns/openshift-dns-operator... 2026-07-13T19:16:36.722256054Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:36.724605696Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-07-13T19:16:36.894955057Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:36.956279656Z Gathering data for ns/openshift-dns... 2026-07-13T19:16:37.081020476Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:37.083619048Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-07-13T19:16:37.261087810Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:37.352728202Z Waiting for performance profile collector pods to become ready: 2 2026-07-13T19:16:37.461035629Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:37.464137313Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-13T19:16:37.637709209Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:37.637762347Z Gathering data for ns/openshift-image-registry... 2026-07-13T19:16:37.833408229Z Wrote inspect data to /must-gather/istio. 2026-07-13T19:16:38.170515250Z Gathering data for ns/openshift-ingress-operator... 2026-07-13T19:16:38.497079510Z Waiting for performance profile collector pods to become ready: 3 2026-07-13T19:16:38.542769334Z Done executing Istio gather script 2026-07-13T19:16:38.582693950Z Gathering data for ns/openshift-ingress... 2026-07-13T19:16:38.936528149Z Gathering data for ns/openshift-ingress-canary... 2026-07-13T19:16:39.322053124Z Gathering data for ns/openshift-insights... 2026-07-13T19:16:39.615535914Z Waiting for performance profile collector pods to become ready: 4 2026-07-13T19:16:40.736182825Z Waiting for performance profile collector pods to become ready: 5 2026-07-13T19:16:41.475477981Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-07-13T19:16:41.550806625Z Gathering data for ns/kserve... 2026-07-13T19:16:41.849367188Z Waiting for performance profile collector pods to become ready: 6 2026-07-13T19:16:42.967044103Z Waiting for performance profile collector pods to become ready: 7 2026-07-13T19:16:43.180579304Z Gathering data for ns/openshift-config... 2026-07-13T19:16:43.484128157Z Gathering data for ns/openshift-config-managed... 2026-07-13T19:16:43.907527803Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-13T19:16:44.079350771Z Waiting for performance profile collector pods to become ready: 8 2026-07-13T19:16:44.221602305Z Gathering data for ns/openshift-kube-apiserver... 2026-07-13T19:16:44.495396275Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-13T19:16:44.781984503Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-13T19:16:45.073362459Z Gathering data for ns/openshift-kube-scheduler... 2026-07-13T19:16:45.207495183Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-13T19:16:45.402944030Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-13T19:16:45.482082297Z Collecting performance related data for node ip-10-0-138-93.ec2.internal 2026-07-13T19:16:45.488714122Z Collecting performance related data for node ip-10-0-134-48.ec2.internal 2026-07-13T19:16:45.495623211Z Collecting performance related data for node ip-10-0-134-215.ec2.internal 2026-07-13T19:16:45.784520792Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-13T19:16:46.257645731Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-13T19:16:46.480170953Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-07-13T19:16:46.639090358Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-13T19:16:47.293888257Z Gathering data for ns/openshift-multus... 2026-07-13T19:16:48.737985255Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-13T19:16:49.998047003Z Gathering data for ns/openshift-host-network... 2026-07-13T19:16:50.316008854Z Gathering data for ns/openshift-network-diagnostics... 2026-07-13T19:16:50.858122446Z Gathering data for ns/openshift-network-node-identity... 2026-07-13T19:16:51.282345541Z Gathering data for ns/openshift-network-operator... 2026-07-13T19:16:51.484685996Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70 2026-07-13T19:16:51.702850376Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-13T19:16:51.976149257Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-13T19:16:52.486982303Z Gathering data for ns/openshift-apiserver-operator... 2026-07-13T19:16:52.792836047Z Gathering data for ns/openshift-apiserver... 2026-07-13T19:16:53.231484572Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-13T19:16:53.513613834Z Gathering data for ns/openshift-controller-manager... 2026-07-13T19:16:53.835921279Z Collecting kubelet logs for node ip-10-0-138-93.ec2.internal 2026-07-13T19:16:53.837734529Z Collecting kubelet logs for node ip-10-0-134-48.ec2.internal 2026-07-13T19:16:53.839347299Z Collecting kubelet logs for node ip-10-0-134-215.ec2.internal 2026-07-13T19:16:53.846614028Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-13T19:16:54.212913393Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-nvxbd namespace 2026-07-13T19:16:54.217806713Z INFO: Node performance data collection complete. 2026-07-13T19:16:54.567780937Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-13T19:16:54.854727877Z Gathering data for ns/openshift-service-ca-operator... 2026-07-13T19:16:55.179295829Z Gathering data for ns/openshift-service-ca... 2026-07-13T19:16:55.560773810Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-13T19:16:56.437292116Z Wrote inspect data to must-gather. 2026-07-13T19:16:56.437336494Z error: inspection completed with the errors occurred while gathering data: 2026-07-13T19:16:56.437336494Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-13T19:16:56.489053316Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70 2026-07-13T19:16:56.570135228Z error: the server doesn't have a resource type "clusters" 2026-07-13T19:16:56.712397827Z Caches written to disk