2026-07-31T15:42:51.258368972Z [disk usage checker] Started 2026-07-31T15:42:51.262216408Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70 2026-07-31T15:42:51.610240621Z Gathering data for ns/openshift-cluster-version... 2026-07-31T15:42:51.746659119Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-31T15:42:51.837962613Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-31T15:42:51.922005531Z Gathering data for ns/default... 2026-07-31T15:42:52.230231232Z Gathering data for ns/openshift... 2026-07-31T15:42:52.654871468Z Gathering data for ns/kube-system... 2026-07-31T15:42:53.406159948Z Gathering data for ns/openshift-etcd... 2026-07-31T15:42:53.816924705Z Wrote inspect data to must-gather. 2026-07-31T15:42:53.817030601Z error: inspection completed with the errors occurred while gathering data: 2026-07-31T15:42:53.817030601Z namespaces "assisted-installer" not found 2026-07-31T15:42:54.323357568Z Waiting on subprocesses to finish execution. 2026-07-31T15:42:54.336355458Z Executing Istio gather script 2026-07-31T15:42:54.337178909Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-31T15:42:54.337809137Z INFO: Gathering HAProxy config files 2026-07-31T15:42:54.338864435Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-31T15:42:54.344295679Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-31T15:42:54.355957811Z INFO: Waiting for node performance related collection to complete ... 2026-07-31T15:42:54.463983710Z INFO: Collecting host service logs for crio 2026-07-31T15:42:54.464359536Z INFO: Collecting host service logs for kubelet 2026-07-31T15:42:54.464701980Z INFO: Collecting host service logs for rpm-ostreed 2026-07-31T15:42:54.465062515Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-31T15:42:54.465410275Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-31T15:42:54.465753541Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-31T15:42:54.466091550Z INFO: Collecting host service logs for NetworkManager 2026-07-31T15:42:54.466397702Z INFO: Collecting host service logs for openvswitch 2026-07-31T15:42:54.466765704Z INFO: Collecting host service logs for ovs-configuration 2026-07-31T15:42:54.467126075Z INFO: Collecting host service logs for ovsdb-server 2026-07-31T15:42:54.467477566Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-31T15:42:54.467911530Z INFO: Waiting for worker host service log collection to complete ... 2026-07-31T15:42:55.102388572Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-31T15:42:55.120039108Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-31T15:42:55.120079609Z INFO: on-disk MachineConfig config collection complete. 2026-07-31T15:42:55.279572859Z error: the server doesn't have a resource type "performanceprofile" 2026-07-31T15:42:55.332854651Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-31T15:42:55.356688506Z INFO: Collecting Insights Archives from insights-operator-77696685b-cx286 2026-07-31T15:42:55.356775002Z insights-runtime-extractor-4b8hs 2026-07-31T15:42:55.356801497Z insights-runtime-extractor-dnt6w 2026-07-31T15:42:55.356823243Z insights-runtime-extractor-qrnb5 2026-07-31T15:42:55.397766499Z No resources found 2026-07-31T15:42:55.463837945Z No resources found in openshift-etcd namespace. 2026-07-31T15:42:55.733561445Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-31T15:42:56.007949362Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-31T15:42:56.132329045Z INFO: "metallb-operator" not detected. Skipping. 2026-07-31T15:42:56.269623312Z Wrote inspect data to must-gather. 2026-07-31T15:42:56.329325435Z No resources found 2026-07-31T15:42:56.391921687Z Wrote inspect data to must-gather. 2026-07-31T15:42:56.403169761Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70 2026-07-31T15:42:56.421212121Z Wrote inspect data to must-gather. 2026-07-31T15:42:56.451065945Z Wrote inspect data to must-gather. 2026-07-31T15:42:56.505509609Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-31T15:42:56.546134378Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-31T15:42:56.590303194Z Wrote inspect data to must-gather. 2026-07-31T15:42:56.619071856Z INFO: Worker host service log collection to complete. 2026-07-31T15:42:56.695308883Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-31T15:42:56.805191643Z ERROR: No running kube-apiserver pods found 2026-07-31T15:42:56.806867788Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-31T15:42:56.841710722Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-31T15:42:56.841710722Z See 'oc rsync -h' for help and examples 2026-07-31T15:42:56.866979233Z Wrote inspect data to must-gather. 2026-07-31T15:42:56.896038288Z Wrote inspect data to must-gather. 2026-07-31T15:42:56.913047610Z INFO: 'previous-logs' folder not found on ip-10-0-129-192.ec2.internal, skipping... 2026-07-31T15:42:57.068090330Z error: the server doesn't have a resource type "machineconfigs" 2026-07-31T15:42:57.075316395Z tar: Removing leading `/' from member names 2026-07-31T15:42:57.081351505Z INFO: HAProxy config collection complete. 2026-07-31T15:42:57.165364246Z Wrote inspect data to must-gather. 2026-07-31T15:42:57.248321191Z Gathering data for ns/openshift-monitoring... 2026-07-31T15:42:57.292118630Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-31T15:42:57.304768762Z INFO: 'previous-logs' folder not found on ip-10-0-130-97.ec2.internal, skipping... 2026-07-31T15:42:57.413334250Z INFO: Getting rules from prometheus-k8s-0 2026-07-31T15:42:57.498362638Z Wrote inspect data to must-gather. 2026-07-31T15:42:57.596079553Z Wrote inspect data to must-gather. 2026-07-31T15:42:57.626708702Z INFO: 'previous-logs' folder not found on ip-10-0-142-219.ec2.internal, skipping... 2026-07-31T15:42:57.626773066Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-31T15:42:57.626773066Z INFO: Machine Config Daemon termination log collection complete. 2026-07-31T15:42:57.791356396Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-31T15:42:57.799933858Z Wrote inspect data to must-gather. 2026-07-31T15:42:57.904351450Z INFO: Getting status/config from prometheus-k8s-0 2026-07-31T15:42:58.054930420Z Wrote inspect data to must-gather. 2026-07-31T15:42:58.150417239Z INFO: INTERCONNECT MODE 2026-07-31T15:42:58.150472434Z INFO: Gathering ovn-kubernetes DBs 2026-07-31T15:42:58.314667615Z Wrote inspect data to must-gather. 2026-07-31T15:42:58.350496138Z INFO: Gathering OVN_Northbound from ovnkube-node-k2dtc... 2026-07-31T15:42:58.350860403Z INFO: Gathering OVN_Northbound from ovnkube-node-qp8cd... 2026-07-31T15:42:58.351097575Z INFO: Gathering OVN_Northbound from ovnkube-node-wp76d... 2026-07-31T15:42:58.351568322Z INFO: Gathering OVN_Southbound from ovnkube-node-k2dtc... 2026-07-31T15:42:58.351861570Z INFO: Gathering OVN_Southbound from ovnkube-node-qp8cd... 2026-07-31T15:42:58.352147774Z INFO: Gathering OVN_Southbound from ovnkube-node-wp76d... 2026-07-31T15:42:58.446514358Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-31T15:42:58.496470906Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-31T15:42:58.680037278Z tar: Removing leading `/' from member names 2026-07-31T15:42:58.688896211Z Wrote inspect data to must-gather. 2026-07-31T15:42:58.722332005Z tar: Removing leading `/' from member names 2026-07-31T15:42:58.736016213Z tar: Removing leading `/' from member names 2026-07-31T15:42:58.739451637Z tar: Removing leading `/' from member names 2026-07-31T15:42:58.742824242Z tar: Removing leading `/' from member names 2026-07-31T15:42:58.777152962Z tar: Removing leading `/' from member names 2026-07-31T15:42:58.815280510Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-31T15:42:58.987389758Z Wrote inspect data to must-gather. 2026-07-31T15:42:59.081506865Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-31T15:42:59.094422536Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-31T15:42:59.098557912Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-31T15:42:59.253025337Z Wrote inspect data to must-gather. 2026-07-31T15:42:59.317120375Z INFO: Waiting for network log collection to complete ... 2026-07-31T15:42:59.317154167Z INFO: Waiting for ovnk database copies to complete ... 2026-07-31T15:42:59.317534508Z INFO: Copying ovnk databases complete. 2026-07-31T15:42:59.319242577Z 4.6M must-gather/network_logs/ovnk_database_store 2026-07-31T15:42:59.321875722Z ovnk_database_store/ 2026-07-31T15:42:59.321930550Z ovnk_database_store/ovnkube-node-wp76d_sbdb 2026-07-31T15:42:59.322425022Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:42:59.350536572Z ovnk_database_store/ovnkube-node-k2dtc_nbdb 2026-07-31T15:42:59.363878034Z ovnk_database_store/ovnkube-node-wp76d_nbdb 2026-07-31T15:42:59.373529221Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-31T15:42:59.378133488Z ovnk_database_store/ovnkube-node-qp8cd_sbdb 2026-07-31T15:42:59.406543311Z ovnk_database_store/ovnkube-node-k2dtc_sbdb 2026-07-31T15:42:59.436274289Z ovnk_database_store/ovnkube-node-qp8cd_nbdb 2026-07-31T15:42:59.452701829Z INFO: Network log collection complete. 2026-07-31T15:42:59.534107168Z Wrote inspect data to must-gather. 2026-07-31T15:42:59.567401817Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:42:59.572347066Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-31T15:42:59.774622320Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-31T15:42:59.779370261Z INFO: Getting status from alertmanager-main-0 2026-07-31T15:42:59.807854375Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:42:59.860054198Z Wrote inspect data to must-gather. 2026-07-31T15:42:59.995660578Z Gathering data for ns/openshift-network-console... 2026-07-31T15:43:00.034527585Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:00.038747064Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-31T15:43:00.161759106Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-31T15:43:00.164520825Z INFO: Fallback to identify the container image from release info 2026-07-31T15:43:00.255897113Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:00.449442516Z Gathering data for ns/openshift-console-operator... 2026-07-31T15:43:00.472789118Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:00.476675406Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-31T15:43:00.700635041Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:00.854491723Z Gathering data for ns/openshift-console... 2026-07-31T15:43:00.866791548Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675 2026-07-31T15:43:00.913386758Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:00.916437986Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-07-31T15:43:01.020468013Z daemonset.apps/perf-node-gather-daemonset created 2026-07-31T15:43:01.109083267Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:01.162283493Z Waiting for performance profile collector pods to become ready: 1 2026-07-31T15:43:01.321698765Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:01.324892781Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-07-31T15:43:01.456593112Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-31T15:43:01.470272622Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70 2026-07-31T15:43:01.524289894Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:01.743429863Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:01.747222250Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-31T15:43:01.928674596Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:02.033591695Z Gathering data for ns/openshift-dns-operator... 2026-07-31T15:43:02.157364686Z Wrote inspect data to /must-gather/istio. 2026-07-31T15:43:02.311108127Z Waiting for performance profile collector pods to become ready: 2 2026-07-31T15:43:02.399800036Z Gathering data for ns/openshift-dns... 2026-07-31T15:43:02.877341359Z Done executing Istio gather script 2026-07-31T15:43:03.047992072Z Gathering data for ns/openshift-image-registry... 2026-07-31T15:43:03.447190263Z Waiting for performance profile collector pods to become ready: 3 2026-07-31T15:43:03.549157649Z Gathering data for ns/openshift-ingress-operator... 2026-07-31T15:43:03.875033991Z Gathering data for ns/openshift-ingress... 2026-07-31T15:43:04.209017359Z Gathering data for ns/openshift-ingress-canary... 2026-07-31T15:43:04.588276987Z Waiting for performance profile collector pods to become ready: 4 2026-07-31T15:43:04.606245946Z Gathering data for ns/openshift-insights... 2026-07-31T15:43:05.707147230Z Waiting for performance profile collector pods to become ready: 5 2026-07-31T15:43:06.475099781Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70 2026-07-31T15:43:06.829547601Z Waiting for performance profile collector pods to become ready: 6 2026-07-31T15:43:07.130832466Z Gathering data for ns/kserve... 2026-07-31T15:43:07.958510710Z Waiting for performance profile collector pods to become ready: 7 2026-07-31T15:43:09.084398486Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-31T15:43:09.319907912Z Gathering data for ns/openshift-config... 2026-07-31T15:43:09.364873773Z Collecting performance related data for node ip-10-0-129-192.ec2.internal 2026-07-31T15:43:09.375141155Z Collecting performance related data for node ip-10-0-130-97.ec2.internal 2026-07-31T15:43:09.386009189Z Collecting performance related data for node ip-10-0-142-219.ec2.internal 2026-07-31T15:43:09.652659321Z Gathering data for ns/openshift-config-managed... 2026-07-31T15:43:10.166956303Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-31T15:43:10.495678881Z Gathering data for ns/openshift-kube-apiserver... 2026-07-31T15:43:10.846283239Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-31T15:43:11.218950943Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-31T15:43:11.481152776Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70 2026-07-31T15:43:11.513840027Z Gathering data for ns/openshift-kube-scheduler... 2026-07-31T15:43:11.825608245Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-31T15:43:12.142188732Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-31T15:43:12.478280723Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-31T15:43:12.800328139Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-31T15:43:13.334648316Z Gathering data for ns/openshift-multus... 2026-07-31T15:43:14.806347903Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-31T15:43:16.134817165Z Gathering data for ns/openshift-host-network... 2026-07-31T15:43:16.486005574Z Gathering data for ns/openshift-network-diagnostics... 2026-07-31T15:43:16.486163246Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70 2026-07-31T15:43:17.085519898Z Gathering data for ns/openshift-network-node-identity... 2026-07-31T15:43:17.534511030Z Gathering data for ns/openshift-network-operator... 2026-07-31T15:43:17.963716938Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-31T15:43:18.236192780Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-31T15:43:18.727776325Z Gathering data for ns/openshift-apiserver-operator... 2026-07-31T15:43:18.779931637Z Collecting kubelet logs for node ip-10-0-129-192.ec2.internal 2026-07-31T15:43:18.782214866Z Collecting kubelet logs for node ip-10-0-130-97.ec2.internal 2026-07-31T15:43:18.784415785Z Collecting kubelet logs for node ip-10-0-142-219.ec2.internal 2026-07-31T15:43:19.138603643Z Gathering data for ns/openshift-apiserver... 2026-07-31T15:43:19.254639765Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-cxx9f namespace 2026-07-31T15:43:19.260595941Z INFO: Node performance data collection complete. 2026-07-31T15:43:19.587894637Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-31T15:43:19.864444179Z Gathering data for ns/openshift-controller-manager... 2026-07-31T15:43:20.178802583Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-31T15:43:20.905500933Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-31T15:43:21.180496983Z Gathering data for ns/openshift-service-ca-operator... 2026-07-31T15:43:21.490979361Z [disk usage checker] Volume usage percentage: current = 40 ; allowed = 70 2026-07-31T15:43:21.496526020Z Gathering data for ns/openshift-service-ca... 2026-07-31T15:43:21.853193255Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-31T15:43:22.742467856Z Wrote inspect data to must-gather. 2026-07-31T15:43:22.742517182Z error: inspection completed with the errors occurred while gathering data: 2026-07-31T15:43:22.742517182Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-31T15:43:22.879430899Z error: the server doesn't have a resource type "clusters" 2026-07-31T15:43:23.036826299Z Caches written to disk