2026-07-15T17:00:15.850286024Z [disk usage checker] Started 2026-07-15T17:00:15.854067495Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-15T17:00:16.052242377Z Gathering data for ns/openshift-cluster-version... 2026-07-15T17:00:16.189864736Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-15T17:00:16.285983986Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-15T17:00:16.370835433Z Gathering data for ns/default... 2026-07-15T17:00:16.719355947Z Gathering data for ns/openshift... 2026-07-15T17:00:17.127261270Z Gathering data for ns/kube-system... 2026-07-15T17:00:17.895586312Z Gathering data for ns/openshift-etcd... 2026-07-15T17:00:18.281933135Z Wrote inspect data to must-gather. 2026-07-15T17:00:18.281966211Z error: inspection completed with the errors occurred while gathering data: 2026-07-15T17:00:18.281966211Z namespaces "assisted-installer" not found 2026-07-15T17:00:18.475411731Z Waiting on subprocesses to finish execution. 2026-07-15T17:00:18.487692828Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-15T17:00:18.490804373Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-15T17:00:18.494738905Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-15T17:00:18.498164853Z INFO: Collecting host service logs for crio 2026-07-15T17:00:18.498693469Z INFO: Collecting host service logs for kubelet 2026-07-15T17:00:18.499055123Z INFO: Collecting host service logs for rpm-ostreed 2026-07-15T17:00:18.499096010Z INFO: Gathering HAProxy config files 2026-07-15T17:00:18.499401160Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-15T17:00:18.499769278Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-15T17:00:18.500122302Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-15T17:00:18.500458654Z INFO: Collecting host service logs for NetworkManager 2026-07-15T17:00:18.500806486Z INFO: Collecting host service logs for openvswitch 2026-07-15T17:00:18.501168406Z INFO: Collecting host service logs for ovs-configuration 2026-07-15T17:00:18.501533641Z INFO: Collecting host service logs for ovsdb-server 2026-07-15T17:00:18.501868515Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-15T17:00:18.502276971Z INFO: Waiting for worker host service log collection to complete ... 2026-07-15T17:00:18.504507913Z Executing Istio gather script 2026-07-15T17:00:18.517845391Z INFO: Waiting for node performance related collection to complete ... 2026-07-15T17:00:19.004141130Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-15T17:00:19.051358924Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-15T17:00:19.224160487Z No resources found 2026-07-15T17:00:19.271058788Z INFO: "metallb-operator" not detected. Skipping. 2026-07-15T17:00:19.438265197Z No resources found in openshift-etcd namespace. 2026-07-15T17:00:19.463330053Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-kxmx7 2026-07-15T17:00:19.463330053Z insights-runtime-extractor-9r45q 2026-07-15T17:00:19.463330053Z insights-runtime-extractor-9wf8d 2026-07-15T17:00:19.463330053Z insights-runtime-extractor-g8ftf 2026-07-15T17:00:19.670892775Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-15T17:00:19.670937596Z INFO: on-disk MachineConfig config collection complete. 2026-07-15T17:00:19.699737963Z ERROR: No running kube-apiserver pods found 2026-07-15T17:00:19.741124299Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-15T17:00:19.806677275Z error: the server doesn't have a resource type "performanceprofile" 2026-07-15T17:00:19.825413979Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-15T17:00:19.825413979Z See 'oc rsync -h' for help and examples 2026-07-15T17:00:19.998232802Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-15T17:00:20.174573790Z Wrote inspect data to must-gather. 2026-07-15T17:00:20.314459200Z INFO: Worker host service log collection to complete. 2026-07-15T17:00:20.335376780Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-15T17:00:20.494848387Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-15T17:00:20.506637651Z Wrote inspect data to must-gather. 2026-07-15T17:00:20.593743417Z Gathering data for ns/openshift-monitoring... 2026-07-15T17:00:20.711146551Z No resources found 2026-07-15T17:00:20.820801118Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-15T17:00:20.843217394Z Wrote inspect data to must-gather. 2026-07-15T17:00:20.845549885Z Wrote inspect data to must-gather. 2026-07-15T17:00:20.874853748Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-15T17:00:20.964195633Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-15T17:00:20.995843059Z Wrote inspect data to must-gather. 2026-07-15T17:00:21.003913323Z Wrote inspect data to must-gather. 2026-07-15T17:00:21.055018286Z INFO: 'previous-logs' folder not found on ip-10-0-134-57.ec2.internal, skipping... 2026-07-15T17:00:21.099298389Z Wrote inspect data to must-gather. 2026-07-15T17:00:21.270574008Z tar: Removing leading `/' from member names 2026-07-15T17:00:21.272210739Z error: the server doesn't have a resource type "machineconfigs" 2026-07-15T17:00:21.278617597Z INFO: HAProxy config collection complete. 2026-07-15T17:00:21.281798889Z INFO: Getting rules from prometheus-k8s-0 2026-07-15T17:00:21.308033367Z Wrote inspect data to must-gather. 2026-07-15T17:00:21.388214762Z INFO: 'previous-logs' folder not found on ip-10-0-135-43.ec2.internal, skipping... 2026-07-15T17:00:21.453618619Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-15T17:00:21.474109234Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-15T17:00:21.615021878Z Wrote inspect data to must-gather. 2026-07-15T17:00:21.657353338Z INFO: 'previous-logs' folder not found on ip-10-0-141-31.ec2.internal, skipping... 2026-07-15T17:00:21.657391889Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-15T17:00:21.657415514Z INFO: Machine Config Daemon termination log collection complete. 2026-07-15T17:00:21.701512225Z Wrote inspect data to must-gather. 2026-07-15T17:00:21.718982906Z INFO: Getting status/config from prometheus-k8s-0 2026-07-15T17:00:21.870385941Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-15T17:00:21.872102718Z Wrote inspect data to must-gather. 2026-07-15T17:00:22.097155245Z Wrote inspect data to must-gather. 2026-07-15T17:00:22.153296242Z INFO: INTERCONNECT MODE 2026-07-15T17:00:22.153408231Z INFO: Gathering ovn-kubernetes DBs 2026-07-15T17:00:22.187183217Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-15T17:00:22.313627893Z INFO: Gathering OVN_Northbound from ovnkube-node-7zb2s... 2026-07-15T17:00:22.313967278Z INFO: Gathering OVN_Northbound from ovnkube-node-jj7nl... 2026-07-15T17:00:22.313987948Z Wrote inspect data to must-gather. 2026-07-15T17:00:22.314199764Z INFO: Gathering OVN_Northbound from ovnkube-node-mkc82... 2026-07-15T17:00:22.314534669Z INFO: Gathering OVN_Southbound from ovnkube-node-7zb2s... 2026-07-15T17:00:22.314805981Z INFO: Gathering OVN_Southbound from ovnkube-node-jj7nl... 2026-07-15T17:00:22.315087734Z INFO: Gathering OVN_Southbound from ovnkube-node-mkc82... 2026-07-15T17:00:22.513889273Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-15T17:00:22.597151324Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-15T17:00:22.616310928Z tar: Removing leading `/' from member names 2026-07-15T17:00:22.660298062Z tar: Removing leading `/' from member names 2026-07-15T17:00:22.705774743Z Wrote inspect data to must-gather. 2026-07-15T17:00:22.709117973Z tar: Removing leading `/' from member names 2026-07-15T17:00:22.709304374Z tar: Removing leading `/' from member names 2026-07-15T17:00:22.713616633Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-15T17:00:22.714491484Z tar: Removing leading `/' from member names 2026-07-15T17:00:22.742213818Z tar: Removing leading `/' from member names 2026-07-15T17:00:22.819594389Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:22.851525090Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-15T17:00:22.962408983Z Wrote inspect data to must-gather. 2026-07-15T17:00:23.073748984Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:23.076900113Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-15T17:00:23.136939059Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-15T17:00:23.179924892Z INFO: Waiting for network log collection to complete ... 2026-07-15T17:00:23.180039574Z INFO: Waiting for ovnk database copies to complete ... 2026-07-15T17:00:23.180571605Z INFO: Copying ovnk databases complete. 2026-07-15T17:00:23.182716593Z 2.8M must-gather/network_logs/ovnk_database_store 2026-07-15T17:00:23.185824686Z ovnk_database_store/ 2026-07-15T17:00:23.185877982Z ovnk_database_store/ovnkube-node-7zb2s_nbdb 2026-07-15T17:00:23.196180823Z ovnk_database_store/ovnkube-node-jj7nl_nbdb 2026-07-15T17:00:23.204653819Z ovnk_database_store/ovnkube-node-mkc82_nbdb 2026-07-15T17:00:23.212467603Z ovnk_database_store/ovnkube-node-7zb2s_sbdb 2026-07-15T17:00:23.218829322Z Wrote inspect data to must-gather. 2026-07-15T17:00:23.233019226Z ovnk_database_store/ovnkube-node-mkc82_sbdb 2026-07-15T17:00:23.252534274Z ovnk_database_store/ovnkube-node-jj7nl_sbdb 2026-07-15T17:00:23.281274192Z INFO: Network log collection complete. 2026-07-15T17:00:23.321473919Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:23.378957092Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-15T17:00:23.521303689Z Wrote inspect data to must-gather. 2026-07-15T17:00:23.546140595Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:23.549135328Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-15T17:00:23.559924126Z INFO: Getting status from alertmanager-main-0 2026-07-15T17:00:23.593500147Z Gathering data for ns/openshift-network-console... 2026-07-15T17:00:23.768808240Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:23.813427666Z Wrote inspect data to must-gather. 2026-07-15T17:00:23.980040329Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:23.983442911Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-15T17:00:24.088550956Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-15T17:00:24.090280626Z INFO: Fallback to identify the container image from release info 2026-07-15T17:00:24.165673147Z Gathering data for ns/openshift-console-operator... 2026-07-15T17:00:24.214791314Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:24.414310425Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:24.418308011Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-15T17:00:24.585512998Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:24.591931023Z Gathering data for ns/openshift-console... 2026-07-15T17:00:24.773458131Z Wrote inspect data to /must-gather/istio. 2026-07-15T17:00:25.159642487Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-15T17:00:25.250278178Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d 2026-07-15T17:00:25.381995015Z daemonset.apps/perf-node-gather-daemonset created 2026-07-15T17:00:25.498295156Z Done executing Istio gather script 2026-07-15T17:00:25.525700140Z Waiting for performance profile collector pods to become ready: 1 2026-07-15T17:00:25.749064729Z Gathering data for ns/openshift-dns-operator... 2026-07-15T17:00:25.880287443Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-15T17:00:26.152196927Z Gathering data for ns/openshift-dns... 2026-07-15T17:00:26.649251336Z Waiting for performance profile collector pods to become ready: 2 2026-07-15T17:00:26.795812481Z Gathering data for ns/openshift-image-registry... 2026-07-15T17:00:27.309867705Z Gathering data for ns/openshift-ingress-operator... 2026-07-15T17:00:27.629618718Z Gathering data for ns/openshift-ingress... 2026-07-15T17:00:27.776899595Z Waiting for performance profile collector pods to become ready: 3 2026-07-15T17:00:27.977344637Z Gathering data for ns/openshift-ingress-canary... 2026-07-15T17:00:28.365130554Z Gathering data for ns/openshift-insights... 2026-07-15T17:00:28.894173364Z Waiting for performance profile collector pods to become ready: 4 2026-07-15T17:00:30.003764325Z Waiting for performance profile collector pods to become ready: 5 2026-07-15T17:00:30.253230562Z Gathering data for ns/openshift-jobset-operator... 2026-07-15T17:00:30.885778787Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-15T17:00:31.044931189Z Gathering data for ns/openshift-config... 2026-07-15T17:00:31.129660789Z Waiting for performance profile collector pods to become ready: 6 2026-07-15T17:00:31.356128213Z Gathering data for ns/openshift-config-managed... 2026-07-15T17:00:31.784993919Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-15T17:00:32.110264587Z Gathering data for ns/openshift-kube-apiserver... 2026-07-15T17:00:32.250569041Z Waiting for performance profile collector pods to become ready: 7 2026-07-15T17:00:32.493737077Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-15T17:00:32.773651734Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-15T17:00:33.072506642Z Gathering data for ns/openshift-kube-scheduler... 2026-07-15T17:00:33.342720482Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-15T17:00:33.384293276Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-15T17:00:33.650246779Z Collecting performance related data for node ip-10-0-141-31.ec2.internal 2026-07-15T17:00:33.659909356Z Collecting performance related data for node ip-10-0-134-57.ec2.internal 2026-07-15T17:00:33.669104131Z Collecting performance related data for node ip-10-0-135-43.ec2.internal 2026-07-15T17:00:33.678312893Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-15T17:00:34.078804150Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-15T17:00:34.418686526Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-15T17:00:35.062577555Z Gathering data for ns/openshift-multus... 2026-07-15T17:00:35.893402381Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-15T17:00:36.600389718Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-15T17:00:37.857295303Z Gathering data for ns/openshift-host-network... 2026-07-15T17:00:38.200227685Z Gathering data for ns/openshift-network-diagnostics... 2026-07-15T17:00:38.751013590Z Gathering data for ns/openshift-network-node-identity... 2026-07-15T17:00:39.278422570Z Gathering data for ns/openshift-network-operator... 2026-07-15T17:00:39.728704078Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-15T17:00:39.988006323Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-15T17:00:40.512376853Z Gathering data for ns/openshift-apiserver-operator... 2026-07-15T17:00:40.796148290Z Gathering data for ns/openshift-apiserver... 2026-07-15T17:00:40.898706022Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-15T17:00:41.227699120Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-15T17:00:41.542421604Z Gathering data for ns/openshift-controller-manager... 2026-07-15T17:00:41.827036758Z Collecting kubelet logs for node ip-10-0-141-31.ec2.internal 2026-07-15T17:00:41.829266693Z Collecting kubelet logs for node ip-10-0-134-57.ec2.internal 2026-07-15T17:00:41.831046799Z Collecting kubelet logs for node ip-10-0-135-43.ec2.internal 2026-07-15T17:00:41.897008152Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-15T17:00:42.169037216Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-szxp9 namespace 2026-07-15T17:00:42.175305605Z INFO: Node performance data collection complete. 2026-07-15T17:00:42.665116344Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-15T17:00:43.109896835Z Gathering data for ns/openshift-service-ca-operator... 2026-07-15T17:00:43.540842636Z Gathering data for ns/openshift-service-ca... 2026-07-15T17:00:43.959464408Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-15T17:00:44.897408574Z Wrote inspect data to must-gather. 2026-07-15T17:00:44.897486152Z error: inspection completed with the errors occurred while gathering data: 2026-07-15T17:00:44.897486152Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-15T17:00:45.028772821Z error: the server doesn't have a resource type "clusters" 2026-07-15T17:00:45.145724439Z Caches written to disk