2026-09-17T16:39:15.453871953Z [disk usage checker] Started 2026-09-17T16:39:15.458411762Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-17T16:39:15.668767508Z Gathering data for ns/openshift-cluster-version... 2026-09-17T16:39:15.825796758Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T16:39:15.920867515Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T16:39:16.011314641Z Gathering data for ns/default... 2026-09-17T16:39:16.348121499Z Gathering data for ns/openshift... 2026-09-17T16:39:16.819490034Z Gathering data for ns/kube-system... 2026-09-17T16:39:17.623653084Z Gathering data for ns/openshift-etcd... 2026-09-17T16:39:17.973514098Z Wrote inspect data to must-gather. 2026-09-17T16:39:17.973638152Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T16:39:17.973638152Z namespaces "assisted-installer" not found 2026-09-17T16:39:18.264266635Z Waiting on subprocesses to finish execution. 2026-09-17T16:39:18.276497235Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-17T16:39:18.278023379Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-17T16:39:18.279945501Z INFO: Gathering HAProxy config files 2026-09-17T16:39:18.280923286Z Executing Istio gather script 2026-09-17T16:39:18.282506858Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-17T16:39:18.288254718Z INFO: Collecting host service logs for crio 2026-09-17T16:39:18.288528982Z INFO: Collecting host service logs for kubelet 2026-09-17T16:39:18.288852342Z INFO: Collecting host service logs for rpm-ostreed 2026-09-17T16:39:18.289124308Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-17T16:39:18.289434226Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-17T16:39:18.289792212Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-17T16:39:18.290088607Z INFO: Collecting host service logs for NetworkManager 2026-09-17T16:39:18.290389733Z INFO: Collecting host service logs for openvswitch 2026-09-17T16:39:18.290699719Z INFO: Collecting host service logs for ovs-configuration 2026-09-17T16:39:18.291005167Z INFO: Collecting host service logs for ovsdb-server 2026-09-17T16:39:18.291317612Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-17T16:39:18.291691701Z INFO: Waiting for worker host service log collection to complete ... 2026-09-17T16:39:18.419162351Z INFO: Waiting for node performance related collection to complete ... 2026-09-17T16:39:18.709438426Z No resources found 2026-09-17T16:39:19.191976717Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-17T16:39:19.504522019Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-17T16:39:19.660058105Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-17T16:39:19.724882060Z Wrote inspect data to must-gather. 2026-09-17T16:39:19.783843538Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-17T16:39:19.908183982Z Wrote inspect data to must-gather. 2026-09-17T16:39:19.976711614Z No resources found in openshift-etcd namespace. 2026-09-17T16:39:20.013690412Z INFO: "metallb-operator" not detected. Skipping. 2026-09-17T16:39:20.163828524Z error: the server doesn't have a resource type "performanceprofile" 2026-09-17T16:39:20.272057079Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T16:39:20.275907449Z Wrote inspect data to must-gather. 2026-09-17T16:39:20.384258212Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-wp8vt 2026-09-17T16:39:20.385466911Z insights-runtime-extractor-6ssct 2026-09-17T16:39:20.386255412Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-17T16:39:20.386546138Z insights-runtime-extractor-ctql4 2026-09-17T16:39:20.386546138Z insights-runtime-extractor-ln472 2026-09-17T16:39:20.418092572Z Wrote inspect data to must-gather. 2026-09-17T16:39:20.422817271Z No resources found 2026-09-17T16:39:20.438212089Z INFO: Worker host service log collection to complete. 2026-09-17T16:39:20.467343449Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-17T16:39:20.467422375Z INFO: on-disk MachineConfig config collection complete. 2026-09-17T16:39:20.555826210Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-17T16:39:20.622775895Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-17T16:39:20.635127831Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-17T16:39:20.649178045Z Wrote inspect data to must-gather. 2026-09-17T16:39:20.698361158Z ERROR: No running kube-apiserver pods found 2026-09-17T16:39:20.865582484Z Wrote inspect data to must-gather. 2026-09-17T16:39:20.877568626Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-17T16:39:20.877568626Z See 'oc rsync -h' for help and examples 2026-09-17T16:39:20.925039042Z Gathering data for ns/openshift-monitoring... 2026-09-17T16:39:21.054084142Z INFO: 'previous-logs' folder not found on ip-10-0-130-83.ec2.internal, skipping... 2026-09-17T16:39:21.145209857Z tar: Removing leading `/' from member names 2026-09-17T16:39:21.152426463Z INFO: HAProxy config collection complete. 2026-09-17T16:39:21.162418455Z Wrote inspect data to must-gather. 2026-09-17T16:39:21.274125016Z Wrote inspect data to must-gather. 2026-09-17T16:39:21.368170092Z error: the server doesn't have a resource type "machineconfigs" 2026-09-17T16:39:21.377221449Z INFO: Getting rules from prometheus-k8s-0 2026-09-17T16:39:21.442830463Z INFO: 'previous-logs' folder not found on ip-10-0-131-81.ec2.internal, skipping... 2026-09-17T16:39:21.556816510Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-17T16:39:21.608576694Z Wrote inspect data to must-gather. 2026-09-17T16:39:21.753112541Z INFO: 'previous-logs' folder not found on ip-10-0-135-240.ec2.internal, skipping... 2026-09-17T16:39:21.753148169Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-17T16:39:21.753208539Z INFO: Machine Config Daemon termination log collection complete. 2026-09-17T16:39:21.832360217Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-17T16:39:21.835947715Z Wrote inspect data to must-gather. 2026-09-17T16:39:21.875413584Z INFO: Getting status/config from prometheus-k8s-0 2026-09-17T16:39:21.903989976Z Wrote inspect data to must-gather. 2026-09-17T16:39:22.020315057Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-17T16:39:22.239394500Z INFO: INTERCONNECT MODE 2026-09-17T16:39:22.239435007Z INFO: Gathering ovn-kubernetes DBs 2026-09-17T16:39:22.255297953Z Wrote inspect data to must-gather. 2026-09-17T16:39:22.325150391Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-17T16:39:22.436267391Z INFO: Gathering OVN_Northbound from ovnkube-node-4pvh2... 2026-09-17T16:39:22.436531594Z INFO: Gathering OVN_Northbound from ovnkube-node-q2tbn... 2026-09-17T16:39:22.436788821Z INFO: Gathering OVN_Northbound from ovnkube-node-qrshk... 2026-09-17T16:39:22.437274266Z INFO: Gathering OVN_Southbound from ovnkube-node-4pvh2... 2026-09-17T16:39:22.437568822Z INFO: Gathering OVN_Southbound from ovnkube-node-q2tbn... 2026-09-17T16:39:22.437890681Z INFO: Gathering OVN_Southbound from ovnkube-node-qrshk... 2026-09-17T16:39:22.529222840Z Wrote inspect data to must-gather. 2026-09-17T16:39:22.718129067Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-17T16:39:22.790908740Z tar: Removing leading `/' from member names 2026-09-17T16:39:22.798809374Z tar: Removing leading `/' from member names 2026-09-17T16:39:22.817967680Z tar: Removing leading `/' from member names 2026-09-17T16:39:22.818702994Z tar: Removing leading `/' from member names 2026-09-17T16:39:22.833701042Z tar: Removing leading `/' from member names 2026-09-17T16:39:22.856809266Z tar: Removing leading `/' from member names 2026-09-17T16:39:22.935584822Z Wrote inspect data to must-gather. 2026-09-17T16:39:22.952460170Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T16:39:23.031158043Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-17T16:39:23.216334673Z Wrote inspect data to must-gather. 2026-09-17T16:39:23.328731886Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-17T16:39:23.341382661Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-17T16:39:23.418947539Z INFO: Waiting for network log collection to complete ... 2026-09-17T16:39:23.418985430Z INFO: Waiting for ovnk database copies to complete ... 2026-09-17T16:39:23.419486842Z INFO: Copying ovnk databases complete. 2026-09-17T16:39:23.421837542Z 2.8M must-gather/network_logs/ovnk_database_store 2026-09-17T16:39:23.425690696Z ovnk_database_store/ 2026-09-17T16:39:23.425882647Z ovnk_database_store/ovnkube-node-q2tbn_sbdb 2026-09-17T16:39:23.450229785Z ovnk_database_store/ovnkube-node-4pvh2_nbdb 2026-09-17T16:39:23.458049441Z ovnk_database_store/ovnkube-node-q2tbn_nbdb 2026-09-17T16:39:23.467108276Z ovnk_database_store/ovnkube-node-4pvh2_sbdb 2026-09-17T16:39:23.486480492Z Wrote inspect data to must-gather. 2026-09-17T16:39:23.487684873Z ovnk_database_store/ovnkube-node-qrshk_sbdb 2026-09-17T16:39:23.505286715Z ovnk_database_store/ovnkube-node-qrshk_nbdb 2026-09-17T16:39:23.513740588Z INFO: Network log collection complete. 2026-09-17T16:39:23.562480925Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:23.607210538Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T16:39:23.795138208Z INFO: Getting status from alertmanager-main-0 2026-09-17T16:39:23.805291242Z Wrote inspect data to must-gather. 2026-09-17T16:39:23.815801982Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:23.819906069Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-17T16:39:23.832647852Z Gathering data for ns/openshift-network-console... 2026-09-17T16:39:24.054674437Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:24.096446882Z Wrote inspect data to must-gather. 2026-09-17T16:39:24.262966614Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:24.266753198Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-17T16:39:24.380630147Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-17T16:39:24.382592409Z INFO: Fallback to identify the container image from release info 2026-09-17T16:39:24.420261345Z Gathering data for ns/openshift-console-operator... 2026-09-17T16:39:24.474379433Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:24.687073541Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:24.689568695Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-17T16:39:24.843146176Z Gathering data for ns/openshift-console... 2026-09-17T16:39:24.914795287Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:24.940655604Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea 2026-09-17T16:39:25.079556777Z daemonset.apps/perf-node-gather-daemonset created 2026-09-17T16:39:25.132099428Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:25.134738768Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-17T16:39:25.246427790Z Waiting for performance profile collector pods to become ready: 1 2026-09-17T16:39:25.327973779Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:25.449296597Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-17T16:39:25.547684024Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:25.549850080Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-17T16:39:25.579140598Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-17T16:39:25.721994843Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:25.937230362Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:25.940814788Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-17T16:39:26.085688607Z Gathering data for ns/openshift-dns-operator... 2026-09-17T16:39:26.130628187Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:26.339734319Z Wrote inspect data to /must-gather/istio. 2026-09-17T16:39:26.410056193Z Waiting for performance profile collector pods to become ready: 2 2026-09-17T16:39:26.453591066Z Gathering data for ns/openshift-dns... 2026-09-17T16:39:27.030643393Z Done executing Istio gather script 2026-09-17T16:39:27.163319150Z Gathering data for ns/openshift-image-registry... 2026-09-17T16:39:27.535974477Z Waiting for performance profile collector pods to become ready: 3 2026-09-17T16:39:27.696125273Z Gathering data for ns/openshift-ingress-operator... 2026-09-17T16:39:28.096281144Z Gathering data for ns/openshift-ingress... 2026-09-17T16:39:28.462847879Z Gathering data for ns/openshift-ingress-canary... 2026-09-17T16:39:28.666337851Z Waiting for performance profile collector pods to become ready: 4 2026-09-17T16:39:28.901121697Z Gathering data for ns/openshift-insights... 2026-09-17T16:39:29.781954440Z Waiting for performance profile collector pods to become ready: 5 2026-09-17T16:39:30.584271821Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-17T16:39:30.904361138Z Waiting for performance profile collector pods to become ready: 6 2026-09-17T16:39:31.365062653Z Gathering data for ns/kserve... 2026-09-17T16:39:32.033221704Z Waiting for performance profile collector pods to become ready: 7 2026-09-17T16:39:32.770808753Z Gathering data for ns/openshift-config... 2026-09-17T16:39:33.064505393Z Gathering data for ns/openshift-config-managed... 2026-09-17T16:39:33.174677107Z Waiting for performance profile collector pods to become ready: 8 2026-09-17T16:39:33.544365818Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-17T16:39:33.876838283Z Gathering data for ns/openshift-kube-apiserver... 2026-09-17T16:39:34.203245768Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-17T16:39:34.309279659Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-17T16:39:34.549173273Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-17T16:39:34.586375382Z Collecting performance related data for node ip-10-0-131-81.ec2.internal 2026-09-17T16:39:34.595142376Z Collecting performance related data for node ip-10-0-130-83.ec2.internal 2026-09-17T16:39:34.607308736Z Collecting performance related data for node ip-10-0-135-240.ec2.internal 2026-09-17T16:39:34.932273419Z Gathering data for ns/openshift-kube-scheduler... 2026-09-17T16:39:35.313780273Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-17T16:39:35.591330532Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-17T16:39:35.736105771Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-17T16:39:36.172824837Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-17T16:39:36.524864473Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-17T16:39:37.110989602Z Gathering data for ns/openshift-multus... 2026-09-17T16:39:38.701141505Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-17T16:39:40.140169932Z Gathering data for ns/openshift-host-network... 2026-09-17T16:39:40.484662102Z Gathering data for ns/openshift-network-diagnostics... 2026-09-17T16:39:40.597520760Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-17T16:39:41.066949309Z Gathering data for ns/openshift-network-node-identity... 2026-09-17T16:39:41.597373945Z Gathering data for ns/openshift-network-operator... 2026-09-17T16:39:42.044921888Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-17T16:39:42.386997500Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-17T16:39:42.653442347Z Collecting kubelet logs for node ip-10-0-131-81.ec2.internal 2026-09-17T16:39:42.655517232Z Collecting kubelet logs for node ip-10-0-130-83.ec2.internal 2026-09-17T16:39:42.657292241Z Collecting kubelet logs for node ip-10-0-135-240.ec2.internal 2026-09-17T16:39:43.017958191Z Gathering data for ns/openshift-apiserver-operator... 2026-09-17T16:39:43.032636476Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-rxjpm namespace 2026-09-17T16:39:43.039475492Z INFO: Node performance data collection complete. 2026-09-17T16:39:43.405331333Z Gathering data for ns/openshift-apiserver... 2026-09-17T16:39:43.928631018Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-17T16:39:44.221129991Z Gathering data for ns/openshift-controller-manager... 2026-09-17T16:39:44.524386880Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-17T16:39:45.207254691Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-17T16:39:45.529437148Z Gathering data for ns/openshift-service-ca-operator... 2026-09-17T16:39:45.603877237Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-17T16:39:45.873811851Z Gathering data for ns/openshift-service-ca... 2026-09-17T16:39:46.277917707Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-17T16:39:47.210042937Z Wrote inspect data to must-gather. 2026-09-17T16:39:47.210088565Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T16:39:47.210088565Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-17T16:39:47.347253808Z error: the server doesn't have a resource type "clusters" 2026-09-17T16:39:47.551439583Z Caches written to disk