2026-10-30T12:52:55.239750565Z [disk usage checker] Started 2026-10-30T12:52:55.243750776Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:52:55.629306395Z Gathering data for ns/openshift-cluster-version... 2026-10-30T12:52:55.966345390Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-30T12:52:56.185838869Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-30T12:52:56.386954812Z Gathering data for ns/default... 2026-10-30T12:52:57.095575479Z Gathering data for ns/openshift... 2026-10-30T12:52:57.908676287Z Gathering data for ns/kube-system... 2026-10-30T12:52:58.746683023Z Waiting on subprocesses to finish execution. 2026-10-30T12:52:58.752597089Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-10-30T12:52:58.755026306Z INFO: Gathering machine config daemon's old logs from all nodes 2026-10-30T12:52:58.768615664Z Executing Istio gather script 2026-10-30T12:52:58.774065323Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-10-30T12:52:58.780703186Z INFO: Gathering HAProxy config files 2026-10-30T12:52:58.790655477Z INFO: Collecting host service logs for crio 2026-10-30T12:52:58.790990331Z INFO: Collecting host service logs for kubelet 2026-10-30T12:52:58.791310660Z INFO: Collecting host service logs for rpm-ostreed 2026-10-30T12:52:58.791673015Z INFO: Collecting host service logs for ostree-finalize-staged 2026-10-30T12:52:58.791991513Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-10-30T12:52:58.792327946Z INFO: Collecting host service logs for machine-config-daemon-host 2026-10-30T12:52:58.792678682Z INFO: Collecting host service logs for NetworkManager 2026-10-30T12:52:58.793018319Z INFO: Collecting host service logs for openvswitch 2026-10-30T12:52:58.793347383Z INFO: Collecting host service logs for ovs-configuration 2026-10-30T12:52:58.793703245Z INFO: Collecting host service logs for ovsdb-server 2026-10-30T12:52:58.794043261Z INFO: Collecting host service logs for ovs-vswitchd 2026-10-30T12:52:58.794408908Z INFO: Waiting for worker host service log collection to complete ... 2026-10-30T12:52:58.826136501Z INFO: Waiting for node performance related collection to complete ... 2026-10-30T12:52:59.293617569Z No resources found 2026-10-30T12:52:59.378354471Z No resources found in openshift-etcd namespace. 2026-10-30T12:52:59.409731910Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-8g7q6 2026-10-30T12:52:59.409794651Z insights-runtime-extractor-cbzrg 2026-10-30T12:52:59.409813679Z insights-runtime-extractor-sxdqs 2026-10-30T12:52:59.409831216Z insights-runtime-extractor-zm77x 2026-10-30T12:52:59.774950750Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-10-30T12:52:59.775048418Z INFO: on-disk MachineConfig config collection complete. 2026-10-30T12:52:59.835519487Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-10-30T12:52:59.835519487Z See 'oc rsync -h' for help and examples 2026-10-30T12:52:59.880391541Z INFO: "metallb-operator" not detected. Skipping. 2026-10-30T12:52:59.942001925Z error: the server doesn't have a resource type "performanceprofile" 2026-10-30T12:52:59.998008076Z INFO: "nfd" not detected. Skipping. 2026-10-30T12:53:00.073611516Z INFO: Found 1 replicas - prometheus-k8s-0 2026-10-30T12:53:00.150928012Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-10-30T12:53:00.183557560Z INFO: "sriov-network-operator" not detected. Skipping. 2026-10-30T12:53:00.481409598Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:00.620046146Z Wrote inspect data to must-gather. 2026-10-30T12:53:00.632703003Z ERROR: No running kube-apiserver pods found 2026-10-30T12:53:00.745829173Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-10-30T12:53:00.812210630Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-30T12:53:00.859007530Z INFO: Waiting for HAProxy config collection to complete ... 2026-10-30T12:53:00.880256860Z INFO: Worker host service log collection to complete. 2026-10-30T12:53:00.962661359Z Wrote inspect data to must-gather. 2026-10-30T12:53:01.224930447Z No resources found 2026-10-30T12:53:01.249081241Z Wrote inspect data to must-gather. 2026-10-30T12:53:01.318448415Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-10-30T12:53:01.342294841Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-10-30T12:53:01.424857294Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-10-30T12:53:01.486121668Z Wrote inspect data to must-gather. 2026-10-30T12:53:01.499244422Z Gathering data for ns/openshift-cloud-credential-operator... 2026-10-30T12:53:01.500860062Z Wrote inspect data to must-gather. 2026-10-30T12:53:01.512925815Z INFO: 'previous-logs' folder not found on ip-10-0-135-237.ec2.internal, skipping... 2026-10-30T12:53:01.520523847Z Wrote inspect data to must-gather. 2026-10-30T12:53:01.529104385Z Wrote inspect data to must-gather. 2026-10-30T12:53:01.578566653Z tar: Removing leading `/' from member names 2026-10-30T12:53:01.608421121Z INFO: HAProxy config collection complete. 2026-10-30T12:53:01.630602484Z error: the server doesn't have a resource type "machineconfigs" 2026-10-30T12:53:01.767489817Z Gathering data for ns/openshift-etcd... 2026-10-30T12:53:01.865417545Z error: the server doesn't have a resource type "machineconfigpools" 2026-10-30T12:53:01.891390643Z INFO: 'previous-logs' folder not found on ip-10-0-139-221.ec2.internal, skipping... 2026-10-30T12:53:01.915975881Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-30T12:53:01.957207034Z Wrote inspect data to must-gather. 2026-10-30T12:53:02.180839233Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-30T12:53:02.187625249Z INFO: Getting rules from prometheus-k8s-0 2026-10-30T12:53:02.198157123Z Wrote inspect data to must-gather. 2026-10-30T12:53:02.205330409Z INFO: 'previous-logs' folder not found on ip-10-0-140-71.ec2.internal, skipping... 2026-10-30T12:53:02.205432942Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-10-30T12:53:02.205529248Z INFO: Machine Config Daemon termination log collection complete. 2026-10-30T12:53:02.353064100Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io 2026-10-30T12:53:02.361854215Z Wrote inspect data to must-gather. 2026-10-30T12:53:02.458286292Z error: the server doesn't have a resource type "kubeletconfigs" 2026-10-30T12:53:02.627476538Z Wrote inspect data to must-gather. 2026-10-30T12:53:02.627536408Z error: inspection completed with the errors occurred while gathering data: 2026-10-30T12:53:02.627536408Z namespaces "assisted-installer" not found 2026-10-30T12:53:02.652169823Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:02.713246094Z Wrote inspect data to must-gather. 2026-10-30T12:53:02.748033682Z Wrote inspect data to must-gather. 2026-10-30T12:53:02.902574814Z INFO: Getting status/config from prometheus-k8s-0 2026-10-30T12:53:02.943398035Z Gathering data for ns/openshift-monitoring... 2026-10-30T12:53:02.965082232Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:02.967802731Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-10-30T12:53:03.053080416Z Wrote inspect data to must-gather. 2026-10-30T12:53:03.111611287Z INFO: LEGACY MODE 2026-10-30T12:53:03.111650674Z INFO: Gathering ovn-kubernetes master data 2026-10-30T12:53:03.205009557Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:03.299323102Z Wrote inspect data to must-gather. 2026-10-30T12:53:03.518814697Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:03.523138394Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-10-30T12:53:03.549298047Z INFO: Getting status/flags from prometheus-k8s-0 2026-10-30T12:53:03.597051031Z Wrote inspect data to must-gather. 2026-10-30T12:53:03.818021527Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:03.862202341Z Wrote inspect data to must-gather. 2026-10-30T12:53:03.924579093Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-10-30T12:53:04.082282113Z INFO: Waiting for network log collection to complete ... 2026-10-30T12:53:04.082329325Z INFO: Waiting for ovnk database copies to complete ... 2026-10-30T12:53:04.082882683Z INFO: Copying ovnk databases complete. 2026-10-30T12:53:04.084734753Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory 2026-10-30T12:53:04.085298102Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 699 is not a child of this shell 2026-10-30T12:53:04.085298102Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 700 is not a child of this shell 2026-10-30T12:53:04.085298102Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 701 is not a child of this shell 2026-10-30T12:53:04.085298102Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 702 is not a child of this shell 2026-10-30T12:53:04.085320381Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 703 is not a child of this shell 2026-10-30T12:53:04.085320381Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 704 is not a child of this shell 2026-10-30T12:53:04.085320381Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 705 is not a child of this shell 2026-10-30T12:53:04.085320381Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 706 is not a child of this shell 2026-10-30T12:53:04.085320381Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 716 is not a child of this shell 2026-10-30T12:53:04.085320381Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 717 is not a child of this shell 2026-10-30T12:53:04.085332096Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 718 is not a child of this shell 2026-10-30T12:53:04.085332096Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 719 is not a child of this shell 2026-10-30T12:53:04.085332096Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 721 is not a child of this shell 2026-10-30T12:53:04.089175834Z tar: ovnk_database_store: Cannot stat: No such file or directory 2026-10-30T12:53:04.092709128Z tar: Exiting with failure status due to previous errors 2026-10-30T12:53:04.093363601Z INFO: Network log collection complete. 2026-10-30T12:53:04.128858388Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:04.132644276Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-10-30T12:53:04.173426756Z Wrote inspect data to must-gather. 2026-10-30T12:53:04.287987372Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-10-30T12:53:04.400172465Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:04.406619773Z Wrote inspect data to must-gather. 2026-10-30T12:53:04.671426588Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:04.674450406Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-10-30T12:53:04.703804799Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-10-30T12:53:04.796532522Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod 2026-10-30T12:53:04.960243912Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:04.965457887Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c 2026-10-30T12:53:05.137156357Z daemonset.apps/perf-node-gather-daemonset created 2026-10-30T12:53:05.242733456Z INFO: Getting status from alertmanager-main-0 2026-10-30T12:53:05.261190583Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:05.263883229Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-10-30T12:53:05.302700011Z Waiting for performance profile collector pods to become ready: 1 2026-10-30T12:53:05.472601438Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:05.562494586Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:05.740939091Z Wrote inspect data to /must-gather/istio. 2026-10-30T12:53:06.447969093Z Waiting for performance profile collector pods to become ready: 2 2026-10-30T12:53:06.732387823Z Done executing Istio gather script 2026-10-30T12:53:07.587060953Z Waiting for performance profile collector pods to become ready: 3 2026-10-30T12:53:08.725449866Z Waiting for performance profile collector pods to become ready: 4 2026-10-30T12:53:09.873648341Z Waiting for performance profile collector pods to become ready: 5 2026-10-30T12:53:10.032644846Z Gathering data for ns/openshift-network-console... 2026-10-30T12:53:10.568804225Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:11.022820211Z Waiting for performance profile collector pods to become ready: 6 2026-10-30T12:53:11.031562536Z Gathering data for ns/openshift-console-operator... 2026-10-30T12:53:11.922389344Z Gathering data for ns/openshift-console... 2026-10-30T12:53:12.164782405Z Waiting for performance profile collector pods to become ready: 7 2026-10-30T12:53:13.085215838Z Gathering data for ns/openshift-cluster-storage-operator... 2026-10-30T12:53:13.310070069Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-10-30T12:53:13.631800881Z Collecting performance related data for node ip-10-0-139-221.ec2.internal 2026-10-30T12:53:13.640619568Z Collecting performance related data for node ip-10-0-140-71.ec2.internal 2026-10-30T12:53:13.650051971Z Collecting performance related data for node ip-10-0-135-237.ec2.internal 2026-10-30T12:53:14.331867716Z Gathering data for ns/openshift-dns-operator... 2026-10-30T12:53:15.193816205Z Gathering data for ns/openshift-dns... 2026-10-30T12:53:15.574041376Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:17.332193785Z Gathering data for ns/openshift-image-registry... 2026-10-30T12:53:18.629419683Z Gathering data for ns/openshift-ingress-operator... 2026-10-30T12:53:19.383439350Z Gathering data for ns/openshift-ingress... 2026-10-30T12:53:20.259083287Z Gathering data for ns/openshift-ingress-canary... 2026-10-30T12:53:20.578979877Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:21.418817109Z Gathering data for ns/openshift-insights... 2026-10-30T12:53:22.026902440Z Collecting kubelet logs for node ip-10-0-139-221.ec2.internal 2026-10-30T12:53:22.028864562Z Collecting kubelet logs for node ip-10-0-140-71.ec2.internal 2026-10-30T12:53:22.030827823Z Collecting kubelet logs for node ip-10-0-135-237.ec2.internal 2026-10-30T12:53:22.469654657Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ldbls namespace 2026-10-30T12:53:22.485368661Z INFO: Node performance data collection complete. 2026-10-30T12:53:25.120641528Z Gathering data for ns/openshift-config... 2026-10-30T12:53:25.583625649Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:25.806523714Z Gathering data for ns/openshift-config-managed... 2026-10-30T12:53:26.601095780Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-10-30T12:53:27.265754519Z Gathering data for ns/openshift-kube-apiserver... 2026-10-30T12:53:27.935924019Z Gathering data for ns/openshift-kube-controller-manager... 2026-10-30T12:53:28.657047081Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-10-30T12:53:29.372727443Z Gathering data for ns/openshift-kube-scheduler... 2026-10-30T12:53:30.028405419Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-10-30T12:53:30.588998364Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:30.747035161Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-10-30T12:53:31.686594751Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-10-30T12:53:32.497284798Z Gathering data for ns/openshift-user-workload-monitoring... 2026-10-30T12:53:33.549486725Z Gathering data for ns/openshift-multus... 2026-10-30T12:53:35.593939214Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:38.955688029Z Gathering data for ns/openshift-ovn-kubernetes... 2026-10-30T12:53:40.598717514Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:43.415644317Z Gathering data for ns/openshift-host-network... 2026-10-30T12:53:44.196619428Z Gathering data for ns/openshift-network-diagnostics... 2026-10-30T12:53:45.603587479Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:45.797257037Z Gathering data for ns/openshift-network-node-identity... 2026-10-30T12:53:46.904602667Z Gathering data for ns/openshift-network-operator... 2026-10-30T12:53:48.021982648Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-10-30T12:53:48.679881854Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-10-30T12:53:49.956134511Z Gathering data for ns/openshift-apiserver-operator... 2026-10-30T12:53:50.609601506Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:50.649639486Z Gathering data for ns/openshift-apiserver... 2026-10-30T12:53:51.583331199Z Gathering data for ns/openshift-controller-manager-operator... 2026-10-30T12:53:52.249290281Z Gathering data for ns/openshift-controller-manager... 2026-10-30T12:53:52.942319426Z Gathering data for ns/openshift-cluster-samples-operator... 2026-10-30T12:53:54.226940022Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-10-30T12:53:54.923977843Z Gathering data for ns/openshift-service-ca-operator... 2026-10-30T12:53:55.614648006Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-30T12:53:55.735135590Z Gathering data for ns/openshift-service-ca... 2026-10-30T12:53:56.671706687Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-10-30T12:53:59.145461091Z Wrote inspect data to must-gather. 2026-10-30T12:53:59.145528520Z error: inspection completed with the errors occurred while gathering data: 2026-10-30T12:53:59.145528520Z skipping gathering secrets/support due to error: secrets "support" not found 2026-10-30T12:53:59.343428670Z error: the server doesn't have a resource type "clusters" 2026-10-30T12:53:59.420133089Z Caches written to disk