2026-09-07T13:50:26.349784106Z [disk usage checker] Started 2026-09-07T13:50:26.352992126Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-07T13:50:26.553624014Z Gathering data for ns/openshift-cluster-version... 2026-09-07T13:50:26.703463217Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-07T13:50:26.827380430Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T13:50:26.915565684Z Gathering data for ns/default... 2026-09-07T13:50:27.226446296Z Gathering data for ns/openshift... 2026-09-07T13:50:27.680502524Z Gathering data for ns/kube-system... 2026-09-07T13:50:28.553688632Z Gathering data for ns/openshift-etcd... 2026-09-07T13:50:28.863193037Z Wrote inspect data to must-gather. 2026-09-07T13:50:28.863231342Z error: inspection completed with the errors occurred while gathering data: 2026-09-07T13:50:28.863231342Z namespaces "assisted-installer" not found 2026-09-07T13:50:28.911210449Z Waiting on subprocesses to finish execution. 2026-09-07T13:50:28.917652405Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-07T13:50:28.924635812Z INFO: Gathering HAProxy config files 2026-09-07T13:50:28.932551966Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-07T13:50:28.940035043Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-07T13:50:28.961027295Z INFO: Collecting host service logs for crio 2026-09-07T13:50:28.961381120Z INFO: Collecting host service logs for kubelet 2026-09-07T13:50:28.961681968Z INFO: Collecting host service logs for rpm-ostreed 2026-09-07T13:50:28.961981739Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-07T13:50:28.962282181Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-07T13:50:28.962588111Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-07T13:50:28.962877030Z INFO: Collecting host service logs for NetworkManager 2026-09-07T13:50:28.963160217Z INFO: Collecting host service logs for openvswitch 2026-09-07T13:50:28.963487413Z INFO: Collecting host service logs for ovs-configuration 2026-09-07T13:50:28.963801191Z INFO: Collecting host service logs for ovsdb-server 2026-09-07T13:50:28.964092549Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-07T13:50:28.966608015Z INFO: Waiting for worker host service log collection to complete ... 2026-09-07T13:50:28.966780281Z Executing Istio gather script 2026-09-07T13:50:29.231476455Z INFO: Waiting for node performance related collection to complete ... 2026-09-07T13:50:29.369037542Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-07T13:50:29.414005757Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-892cb 2026-09-07T13:50:29.414078438Z insights-runtime-extractor-8rpbz 2026-09-07T13:50:29.414098427Z insights-runtime-extractor-qzzlj 2026-09-07T13:50:29.414116269Z insights-runtime-extractor-t9rbc 2026-09-07T13:50:29.732879502Z No resources found in openshift-etcd namespace. 2026-09-07T13:50:29.943869341Z No resources found 2026-09-07T13:50:30.128368400Z INFO: "metallb-operator" not detected. Skipping. 2026-09-07T13:50:30.186489327Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-07T13:50:30.194749093Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-07T13:50:30.258395200Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-07T13:50:30.358944611Z ERROR: No running kube-apiserver pods found 2026-09-07T13:50:30.412939108Z error: the server doesn't have a resource type "performanceprofile" 2026-09-07T13:50:30.420626575Z Wrote inspect data to must-gather. 2026-09-07T13:50:30.464802622Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-07T13:50:30.464802622Z See 'oc rsync -h' for help and examples 2026-09-07T13:50:30.526182455Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-07T13:50:30.526182455Z INFO: on-disk MachineConfig config collection complete. 2026-09-07T13:50:30.546377056Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-07T13:50:30.575449795Z INFO: Worker host service log collection to complete. 2026-09-07T13:50:30.909266830Z INFO: 'previous-logs' folder not found on ip-10-0-131-22.ec2.internal, skipping... 2026-09-07T13:50:30.931233024Z Wrote inspect data to must-gather. 2026-09-07T13:50:31.007273035Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-07T13:50:31.088770452Z No resources found 2026-09-07T13:50:31.110616274Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T13:50:31.192809913Z tar: Removing leading `/' from member names 2026-09-07T13:50:31.202910796Z INFO: HAProxy config collection complete. 2026-09-07T13:50:31.266945213Z Wrote inspect data to must-gather. 2026-09-07T13:50:31.358759318Z Gathering data for ns/openshift-monitoring... 2026-09-07T13:50:31.391746841Z [disk usage checker] Volume usage percentage: current = 20 ; allowed = 70 2026-09-07T13:50:31.418960021Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-07T13:50:31.437658871Z Wrote inspect data to must-gather. 2026-09-07T13:50:31.464647383Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-07T13:50:31.488001660Z Wrote inspect data to must-gather. 2026-09-07T13:50:31.539537139Z Wrote inspect data to must-gather. 2026-09-07T13:50:31.541576619Z Wrote inspect data to must-gather. 2026-09-07T13:50:31.639266468Z INFO: 'previous-logs' folder not found on ip-10-0-134-192.ec2.internal, skipping... 2026-09-07T13:50:31.671192380Z error: the server doesn't have a resource type "machineconfigs" 2026-09-07T13:50:31.695412441Z Wrote inspect data to must-gather. 2026-09-07T13:50:31.845482645Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-07T13:50:31.923150819Z INFO: 'previous-logs' folder not found on ip-10-0-140-163.ec2.internal, skipping... 2026-09-07T13:50:31.923255921Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-07T13:50:31.923278379Z INFO: Machine Config Daemon termination log collection complete. 2026-09-07T13:50:31.945378367Z INFO: Getting rules from prometheus-k8s-0 2026-09-07T13:50:31.978078214Z Wrote inspect data to must-gather. 2026-09-07T13:50:32.070404807Z Wrote inspect data to must-gather. 2026-09-07T13:50:32.207428189Z Wrote inspect data to must-gather. 2026-09-07T13:50:32.224848507Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-07T13:50:32.384808741Z INFO: Getting status/config from prometheus-k8s-0 2026-09-07T13:50:32.435416788Z Wrote inspect data to must-gather. 2026-09-07T13:50:32.517922455Z INFO: INTERCONNECT MODE 2026-09-07T13:50:32.517962906Z INFO: Gathering ovn-kubernetes DBs 2026-09-07T13:50:32.651447713Z Wrote inspect data to must-gather. 2026-09-07T13:50:32.669541535Z INFO: Gathering OVN_Northbound from ovnkube-node-4jzxn... 2026-09-07T13:50:32.669768826Z INFO: Gathering OVN_Northbound from ovnkube-node-7knv9... 2026-09-07T13:50:32.669987637Z INFO: Gathering OVN_Northbound from ovnkube-node-dvsp8... 2026-09-07T13:50:32.670270287Z INFO: Gathering OVN_Southbound from ovnkube-node-4jzxn... 2026-09-07T13:50:32.670529710Z INFO: Gathering OVN_Southbound from ovnkube-node-7knv9... 2026-09-07T13:50:32.670831180Z INFO: Gathering OVN_Southbound from ovnkube-node-dvsp8... 2026-09-07T13:50:32.959818644Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-07T13:50:32.985944549Z Wrote inspect data to must-gather. 2026-09-07T13:50:33.029754224Z tar: Removing leading `/' from member names 2026-09-07T13:50:33.089521085Z tar: Removing leading `/' from member names 2026-09-07T13:50:33.097576414Z tar: Removing leading `/' from member names 2026-09-07T13:50:33.110035555Z tar: Removing leading `/' from member names 2026-09-07T13:50:33.125015735Z tar: Removing leading `/' from member names 2026-09-07T13:50:33.146900339Z tar: Removing leading `/' from member names 2026-09-07T13:50:33.162960013Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-07T13:50:33.220701677Z Wrote inspect data to must-gather. 2026-09-07T13:50:33.287298951Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-07T13:50:33.367922746Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:33.468287031Z Wrote inspect data to must-gather. 2026-09-07T13:50:33.522960555Z INFO: Waiting for network log collection to complete ... 2026-09-07T13:50:33.522997010Z INFO: Waiting for ovnk database copies to complete ... 2026-09-07T13:50:33.523420933Z INFO: Copying ovnk databases complete. 2026-09-07T13:50:33.524956405Z 2.8M must-gather/network_logs/ovnk_database_store 2026-09-07T13:50:33.526925496Z ovnk_database_store/ 2026-09-07T13:50:33.526925496Z ovnk_database_store/ovnkube-node-dvsp8_sbdb 2026-09-07T13:50:33.546219355Z ovnk_database_store/ovnkube-node-7knv9_nbdb 2026-09-07T13:50:33.551698531Z ovnk_database_store/ovnkube-node-dvsp8_nbdb 2026-09-07T13:50:33.558025900Z ovnk_database_store/ovnkube-node-4jzxn_sbdb 2026-09-07T13:50:33.560950800Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-07T13:50:33.577330376Z ovnk_database_store/ovnkube-node-7knv9_sbdb 2026-09-07T13:50:33.592554679Z ovnk_database_store/ovnkube-node-4jzxn_nbdb 2026-09-07T13:50:33.603270674Z INFO: Network log collection complete. 2026-09-07T13:50:33.604217390Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:33.608778197Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-07T13:50:33.611182220Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-07T13:50:33.673558846Z Wrote inspect data to must-gather. 2026-09-07T13:50:33.817701811Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:33.876172046Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-07T13:50:33.900108958Z Wrote inspect data to must-gather. 2026-09-07T13:50:34.034477740Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:34.037655505Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-07T13:50:34.186536926Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-07T13:50:34.188672845Z INFO: Fallback to identify the container image from release info 2026-09-07T13:50:34.201295259Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T13:50:34.251999353Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:34.323124184Z INFO: Getting status from alertmanager-main-0 2026-09-07T13:50:34.399787335Z Gathering data for ns/openshift-network-console... 2026-09-07T13:50:34.480671364Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:34.484341031Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-07T13:50:34.689071526Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:34.827990694Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399 2026-09-07T13:50:34.864849494Z Gathering data for ns/openshift-console-operator... 2026-09-07T13:50:34.894162851Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:34.896582605Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-07T13:50:34.961314917Z daemonset.apps/perf-node-gather-daemonset created 2026-09-07T13:50:35.085756479Z Waiting for performance profile collector pods to become ready: 1 2026-09-07T13:50:35.094027862Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:35.283107698Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:50:35.287430284Z Gathering data for ns/openshift-console... 2026-09-07T13:50:35.851395594Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-07T13:50:35.979289046Z Done executing Istio gather script 2026-09-07T13:50:36.205689074Z Waiting for performance profile collector pods to become ready: 2 2026-09-07T13:50:36.381760758Z Gathering data for ns/openshift-dns-operator... 2026-09-07T13:50:36.400301634Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70 2026-09-07T13:50:36.663169629Z Gathering data for ns/openshift-dns... 2026-09-07T13:50:37.326391598Z Waiting for performance profile collector pods to become ready: 3 2026-09-07T13:50:37.444472921Z Gathering data for ns/openshift-image-registry... 2026-09-07T13:50:37.911444117Z Gathering data for ns/openshift-ingress-operator... 2026-09-07T13:50:38.241333312Z Gathering data for ns/openshift-ingress... 2026-09-07T13:50:38.434207230Z Waiting for performance profile collector pods to become ready: 4 2026-09-07T13:50:38.575959186Z Gathering data for ns/openshift-ingress-canary... 2026-09-07T13:50:38.953970382Z Gathering data for ns/openshift-insights... 2026-09-07T13:50:39.557236309Z Waiting for performance profile collector pods to become ready: 5 2026-09-07T13:50:40.676972678Z Waiting for performance profile collector pods to become ready: 6 2026-09-07T13:50:40.924365618Z Gathering data for ns/openshift-jobset-operator... 2026-09-07T13:50:41.405016132Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70 2026-09-07T13:50:41.773107286Z Gathering data for ns/openshift-config... 2026-09-07T13:50:41.788631295Z Waiting for performance profile collector pods to become ready: 7 2026-09-07T13:50:42.068125022Z Gathering data for ns/openshift-config-managed... 2026-09-07T13:50:42.620693849Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-07T13:50:42.910224478Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-07T13:50:42.910996491Z Gathering data for ns/openshift-kube-apiserver... 2026-09-07T13:50:43.164660972Z Collecting performance related data for node ip-10-0-131-22.ec2.internal 2026-09-07T13:50:43.175127155Z Collecting performance related data for node ip-10-0-140-163.ec2.internal 2026-09-07T13:50:43.182810774Z Collecting performance related data for node ip-10-0-134-192.ec2.internal 2026-09-07T13:50:43.236831460Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-07T13:50:43.634121066Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-07T13:50:44.014497243Z Gathering data for ns/openshift-kube-scheduler... 2026-09-07T13:50:44.373257048Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-07T13:50:44.767912882Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-07T13:50:45.203687956Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-07T13:50:45.549614617Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-07T13:50:46.163477222Z Gathering data for ns/openshift-multus... 2026-09-07T13:50:46.409582781Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70 2026-09-07T13:50:48.291085743Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-07T13:50:49.552277262Z Gathering data for ns/openshift-host-network... 2026-09-07T13:50:49.859816134Z Gathering data for ns/openshift-network-diagnostics... 2026-09-07T13:50:50.406409115Z Gathering data for ns/openshift-network-node-identity... 2026-09-07T13:50:50.831502266Z Gathering data for ns/openshift-network-operator... 2026-09-07T13:50:51.235386994Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-07T13:50:51.416109869Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70 2026-09-07T13:50:51.512170130Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-07T13:50:51.558102155Z Collecting kubelet logs for node ip-10-0-131-22.ec2.internal 2026-09-07T13:50:51.559816603Z Collecting kubelet logs for node ip-10-0-140-163.ec2.internal 2026-09-07T13:50:51.561438928Z Collecting kubelet logs for node ip-10-0-134-192.ec2.internal 2026-09-07T13:50:51.911263124Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vzz84 namespace 2026-09-07T13:50:51.919939303Z INFO: Node performance data collection complete. 2026-09-07T13:50:52.173949206Z Gathering data for ns/openshift-apiserver-operator... 2026-09-07T13:50:52.545157979Z Gathering data for ns/openshift-apiserver... 2026-09-07T13:50:53.063024904Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-07T13:50:53.380047725Z Gathering data for ns/openshift-controller-manager... 2026-09-07T13:50:53.643791342Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-07T13:50:54.295196063Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-07T13:50:54.673989306Z Gathering data for ns/openshift-service-ca-operator... 2026-09-07T13:50:54.996576134Z Gathering data for ns/openshift-service-ca... 2026-09-07T13:50:55.355985988Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-07T13:50:56.254303312Z Wrote inspect data to must-gather. 2026-09-07T13:50:56.254375066Z error: inspection completed with the errors occurred while gathering data: 2026-09-07T13:50:56.254375066Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-07T13:50:56.374489055Z error: the server doesn't have a resource type "clusters" 2026-09-07T13:50:56.420810806Z [disk usage checker] Volume usage percentage: current = 21 ; allowed = 70 2026-09-07T13:50:56.513401863Z Caches written to disk