2026-07-28T18:05:37.970858117Z [disk usage checker] Started 2026-07-28T18:05:37.975097896Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-28T18:05:38.183778704Z Gathering data for ns/openshift-cluster-version... 2026-07-28T18:05:38.330564302Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-28T18:05:38.435694905Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T18:05:38.527404946Z Gathering data for ns/default... 2026-07-28T18:05:38.905452941Z Gathering data for ns/openshift... 2026-07-28T18:05:39.409659092Z Gathering data for ns/kube-system... 2026-07-28T18:05:40.272350569Z Gathering data for ns/openshift-etcd... 2026-07-28T18:05:40.660986432Z Wrote inspect data to must-gather. 2026-07-28T18:05:40.661024874Z error: inspection completed with the errors occurred while gathering data: 2026-07-28T18:05:40.661024874Z namespaces "assisted-installer" not found 2026-07-28T18:05:40.749594113Z Waiting on subprocesses to finish execution. 2026-07-28T18:05:40.759568224Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-28T18:05:40.761776334Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-28T18:05:40.765837412Z Executing Istio gather script 2026-07-28T18:05:40.768520035Z INFO: Gathering HAProxy config files 2026-07-28T18:05:40.787447785Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-28T18:05:40.790795495Z INFO: Collecting host service logs for crio 2026-07-28T18:05:40.791197099Z INFO: Collecting host service logs for kubelet 2026-07-28T18:05:40.791554944Z INFO: Collecting host service logs for rpm-ostreed 2026-07-28T18:05:40.791948879Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-28T18:05:40.792297388Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-28T18:05:40.792623955Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-28T18:05:40.793000598Z INFO: Collecting host service logs for NetworkManager 2026-07-28T18:05:40.793357191Z INFO: Collecting host service logs for openvswitch 2026-07-28T18:05:40.793711934Z INFO: Collecting host service logs for ovs-configuration 2026-07-28T18:05:40.794080112Z INFO: Collecting host service logs for ovsdb-server 2026-07-28T18:05:40.794444901Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-28T18:05:40.794870076Z INFO: Waiting for worker host service log collection to complete ... 2026-07-28T18:05:40.863681736Z INFO: Waiting for node performance related collection to complete ... 2026-07-28T18:05:41.276130777Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-28T18:05:41.309314469Z No resources found in openshift-etcd namespace. 2026-07-28T18:05:41.441204806Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-28T18:05:41.441250882Z INFO: on-disk MachineConfig config collection complete. 2026-07-28T18:05:41.602043809Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-ztfm4 2026-07-28T18:05:41.602162959Z insights-runtime-extractor-bzkqv 2026-07-28T18:05:41.602188883Z insights-runtime-extractor-fppjm 2026-07-28T18:05:41.602222892Z insights-runtime-extractor-l5kvz 2026-07-28T18:05:41.677782093Z No resources found 2026-07-28T18:05:41.788774654Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-28T18:05:41.808634066Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-28T18:05:41.894483979Z Wrote inspect data to must-gather. 2026-07-28T18:05:41.922346209Z Wrote inspect data to must-gather. 2026-07-28T18:05:42.094659719Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-28T18:05:42.094659719Z See 'oc rsync -h' for help and examples 2026-07-28T18:05:42.228734730Z error: the server doesn't have a resource type "performanceprofile" 2026-07-28T18:05:42.313082190Z INFO: "metallb-operator" not detected. Skipping. 2026-07-28T18:05:42.364313627Z INFO: Worker host service log collection to complete. 2026-07-28T18:05:42.408739781Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-28T18:05:42.710340298Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-28T18:05:42.902170487Z Wrote inspect data to must-gather. 2026-07-28T18:05:42.994273306Z Gathering data for ns/openshift-monitoring... 2026-07-28T18:05:43.009249111Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-28T18:05:43.014415302Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-28T18:05:43.026010026Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-28T18:05:43.037910277Z ERROR: No running kube-apiserver pods found 2026-07-28T18:05:43.118540371Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T18:05:43.144486960Z INFO: 'previous-logs' folder not found on ip-10-0-138-73.ec2.internal, skipping... 2026-07-28T18:05:43.157157585Z No resources found 2026-07-28T18:05:43.214376628Z Wrote inspect data to must-gather. 2026-07-28T18:05:43.214518305Z Wrote inspect data to must-gather. 2026-07-28T18:05:43.232069997Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-28T18:05:43.260183196Z Wrote inspect data to must-gather. 2026-07-28T18:05:43.311787831Z Wrote inspect data to must-gather. 2026-07-28T18:05:43.410991748Z error: the server doesn't have a resource type "machineconfigs" 2026-07-28T18:05:43.417177136Z tar: Removing leading `/' from member names 2026-07-28T18:05:43.423246670Z INFO: HAProxy config collection complete. 2026-07-28T18:05:43.489432841Z INFO: 'previous-logs' folder not found on ip-10-0-140-55.ec2.internal, skipping... 2026-07-28T18:05:43.580124226Z Wrote inspect data to must-gather. 2026-07-28T18:05:43.601626849Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-28T18:05:43.656030021Z INFO: Getting rules from prometheus-k8s-0 2026-07-28T18:05:43.769055086Z INFO: 'previous-logs' folder not found on ip-10-0-143-32.ec2.internal, skipping... 2026-07-28T18:05:43.769097340Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-28T18:05:43.769129428Z INFO: Machine Config Daemon termination log collection complete. 2026-07-28T18:05:43.842288020Z Wrote inspect data to must-gather. 2026-07-28T18:05:43.844937759Z Wrote inspect data to must-gather. 2026-07-28T18:05:43.996703107Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-28T18:05:44.079603629Z Wrote inspect data to must-gather. 2026-07-28T18:05:44.108960315Z INFO: Getting status/config from prometheus-k8s-0 2026-07-28T18:05:44.222140881Z Wrote inspect data to must-gather. 2026-07-28T18:05:44.360276269Z INFO: INTERCONNECT MODE 2026-07-28T18:05:44.360311463Z INFO: Gathering ovn-kubernetes DBs 2026-07-28T18:05:44.414505519Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-28T18:05:44.415676873Z Wrote inspect data to must-gather. 2026-07-28T18:05:44.524284595Z INFO: Gathering OVN_Northbound from ovnkube-node-69nlc... 2026-07-28T18:05:44.524333576Z INFO: Gathering OVN_Northbound from ovnkube-node-lkhgf... 2026-07-28T18:05:44.524557174Z INFO: Gathering OVN_Northbound from ovnkube-node-vq4x8... 2026-07-28T18:05:44.524824790Z INFO: Gathering OVN_Southbound from ovnkube-node-69nlc... 2026-07-28T18:05:44.525086008Z INFO: Gathering OVN_Southbound from ovnkube-node-lkhgf... 2026-07-28T18:05:44.525334204Z INFO: Gathering OVN_Southbound from ovnkube-node-vq4x8... 2026-07-28T18:05:44.593987864Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-28T18:05:44.657741593Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:44.752395702Z Wrote inspect data to must-gather. 2026-07-28T18:05:44.859996646Z tar: Removing leading `/' from member names 2026-07-28T18:05:44.873166140Z tar: Removing leading `/' from member names 2026-07-28T18:05:44.903997544Z tar: Removing leading `/' from member names 2026-07-28T18:05:44.908280236Z tar: Removing leading `/' from member names 2026-07-28T18:05:44.915635390Z tar: Removing leading `/' from member names 2026-07-28T18:05:44.927092417Z tar: Removing leading `/' from member names 2026-07-28T18:05:44.992462624Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:44.996025019Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-28T18:05:45.019761175Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-28T18:05:45.042718970Z Wrote inspect data to must-gather. 2026-07-28T18:05:45.141733910Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-28T18:05:45.259355809Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:45.281255740Z Wrote inspect data to must-gather. 2026-07-28T18:05:45.290990946Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-28T18:05:45.390877044Z INFO: Waiting for network log collection to complete ... 2026-07-28T18:05:45.390943924Z INFO: Waiting for ovnk database copies to complete ... 2026-07-28T18:05:45.391457195Z INFO: Copying ovnk databases complete. 2026-07-28T18:05:45.393600449Z 2.4M must-gather/network_logs/ovnk_database_store 2026-07-28T18:05:45.396795745Z ovnk_database_store/ 2026-07-28T18:05:45.396861427Z ovnk_database_store/ovnkube-node-69nlc_sbdb 2026-07-28T18:05:45.415844209Z ovnk_database_store/ovnkube-node-vq4x8_nbdb 2026-07-28T18:05:45.422265081Z ovnk_database_store/ovnkube-node-lkhgf_sbdb 2026-07-28T18:05:45.441387725Z ovnk_database_store/ovnkube-node-69nlc_nbdb 2026-07-28T18:05:45.448250019Z ovnk_database_store/ovnkube-node-vq4x8_sbdb 2026-07-28T18:05:45.463648579Z ovnk_database_store/ovnkube-node-lkhgf_nbdb 2026-07-28T18:05:45.473873854Z INFO: Network log collection complete. 2026-07-28T18:05:45.495259998Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:45.498392732Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-28T18:05:45.528192815Z Wrote inspect data to must-gather. 2026-07-28T18:05:45.578459019Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-28T18:05:45.708125643Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:45.735547833Z W0728 18:05:45.735491 1523 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-28T18:05:45.740006864Z Wrote inspect data to must-gather. 2026-07-28T18:05:45.865460949Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-28T18:05:45.947054791Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:45.949482750Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-28T18:05:46.033264864Z INFO: Getting status from alertmanager-main-0 2026-07-28T18:05:46.035750660Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-28T18:05:46.039133883Z INFO: Fallback to identify the container image from release info 2026-07-28T18:05:46.086313252Z Gathering data for ns/openshift-network-console... 2026-07-28T18:05:46.204549889Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:46.394055448Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:46.396584007Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-28T18:05:46.569608814Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:46.631800380Z Gathering data for ns/openshift-console-operator... 2026-07-28T18:05:46.770575690Z Wrote inspect data to /must-gather/istio. 2026-07-28T18:05:47.130454704Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811 2026-07-28T18:05:47.186479562Z Gathering data for ns/openshift-console... 2026-07-28T18:05:47.265569736Z daemonset.apps/perf-node-gather-daemonset created 2026-07-28T18:05:47.406289089Z Waiting for performance profile collector pods to become ready: 1 2026-07-28T18:05:47.464687461Z Done executing Istio gather script 2026-07-28T18:05:47.912228240Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-28T18:05:48.096581299Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-28T18:05:48.468979570Z Gathering data for ns/openshift-dns-operator... 2026-07-28T18:05:48.541490507Z Waiting for performance profile collector pods to become ready: 2 2026-07-28T18:05:48.789378575Z Gathering data for ns/openshift-dns... 2026-07-28T18:05:49.480705598Z Gathering data for ns/openshift-image-registry... 2026-07-28T18:05:49.661864315Z Waiting for performance profile collector pods to become ready: 3 2026-07-28T18:05:50.057270546Z Gathering data for ns/openshift-ingress-operator... 2026-07-28T18:05:50.485503391Z Gathering data for ns/openshift-ingress... 2026-07-28T18:05:50.786346804Z Waiting for performance profile collector pods to become ready: 4 2026-07-28T18:05:50.895604067Z Gathering data for ns/openshift-ingress-canary... 2026-07-28T18:05:51.413039589Z Gathering data for ns/openshift-insights... 2026-07-28T18:05:51.897283055Z Waiting for performance profile collector pods to become ready: 5 2026-07-28T18:05:53.019395151Z Waiting for performance profile collector pods to become ready: 6 2026-07-28T18:05:53.101543696Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-28T18:05:53.462492823Z Gathering data for ns/openshift-config... 2026-07-28T18:05:53.804187727Z Gathering data for ns/openshift-config-managed... 2026-07-28T18:05:54.136733275Z Waiting for performance profile collector pods to become ready: 7 2026-07-28T18:05:54.276267613Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-28T18:05:54.594062771Z Gathering data for ns/openshift-kube-apiserver... 2026-07-28T18:05:54.884396301Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-28T18:05:55.210091632Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-28T18:05:55.272077053Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-28T18:05:55.522302444Z Collecting performance related data for node ip-10-0-143-32.ec2.internal 2026-07-28T18:05:55.533829428Z Collecting performance related data for node ip-10-0-138-73.ec2.internal 2026-07-28T18:05:55.543793117Z Collecting performance related data for node ip-10-0-140-55.ec2.internal 2026-07-28T18:05:55.595302411Z Gathering data for ns/openshift-kube-scheduler... 2026-07-28T18:05:55.973428328Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-28T18:05:56.385135884Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-28T18:05:56.831852899Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-28T18:05:57.214009602Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-28T18:05:57.794727830Z Gathering data for ns/openshift-multus... 2026-07-28T18:05:58.106768763Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-28T18:05:59.633194875Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-28T18:06:01.162761360Z Gathering data for ns/openshift-host-network... 2026-07-28T18:06:01.564408498Z Gathering data for ns/openshift-network-diagnostics... 2026-07-28T18:06:02.208463628Z Gathering data for ns/openshift-network-node-identity... 2026-07-28T18:06:02.702498049Z Gathering data for ns/openshift-network-operator... 2026-07-28T18:06:03.112484806Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-28T18:06:03.191528387Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-28T18:06:03.564420496Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-28T18:06:03.656941384Z Collecting kubelet logs for node ip-10-0-143-32.ec2.internal 2026-07-28T18:06:03.658880129Z Collecting kubelet logs for node ip-10-0-138-73.ec2.internal 2026-07-28T18:06:03.660874252Z Collecting kubelet logs for node ip-10-0-140-55.ec2.internal 2026-07-28T18:06:04.017591741Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-qd9rn namespace 2026-07-28T18:06:04.029201402Z INFO: Node performance data collection complete. 2026-07-28T18:06:04.203176304Z Gathering data for ns/openshift-apiserver-operator... 2026-07-28T18:06:04.603175511Z Gathering data for ns/openshift-apiserver... 2026-07-28T18:06:05.052197733Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-28T18:06:05.353536667Z Gathering data for ns/openshift-controller-manager... 2026-07-28T18:06:05.698534775Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-28T18:06:06.506808686Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-28T18:06:06.883609634Z Gathering data for ns/openshift-service-ca-operator... 2026-07-28T18:06:07.292672799Z Gathering data for ns/openshift-service-ca... 2026-07-28T18:06:07.790511893Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-28T18:06:08.117489672Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-07-28T18:06:08.926828728Z Wrote inspect data to must-gather. 2026-07-28T18:06:08.926886064Z error: inspection completed with the errors occurred while gathering data: 2026-07-28T18:06:08.926886064Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-28T18:06:09.062934962Z error: the server doesn't have a resource type "clusters" 2026-07-28T18:06:09.195114648Z Caches written to disk