2026-09-07T13:27:04.368363742Z [disk usage checker] Started 2026-09-07T13:27:04.371905200Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-07T13:27:04.589611215Z Gathering data for ns/openshift-cluster-version... 2026-09-07T13:27:04.754369298Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-07T13:27:04.873412412Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T13:27:04.961452189Z Gathering data for ns/default... 2026-09-07T13:27:05.296261392Z Gathering data for ns/openshift... 2026-09-07T13:27:05.737829389Z Gathering data for ns/kube-system... 2026-09-07T13:27:06.555979863Z Gathering data for ns/openshift-etcd... 2026-09-07T13:27:06.931470966Z Wrote inspect data to must-gather. 2026-09-07T13:27:06.931504755Z error: inspection completed with the errors occurred while gathering data: 2026-09-07T13:27:06.931504755Z namespaces "assisted-installer" not found 2026-09-07T13:27:06.972482799Z Waiting on subprocesses to finish execution. 2026-09-07T13:27:06.977385592Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-07T13:27:06.991408768Z INFO: Collecting host service logs for crio 2026-09-07T13:27:06.991776378Z INFO: Collecting host service logs for kubelet 2026-09-07T13:27:06.992117986Z INFO: Collecting host service logs for rpm-ostreed 2026-09-07T13:27:06.992436075Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-07T13:27:06.992797094Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-07T13:27:06.993128051Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-07T13:27:06.993379612Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-07T13:27:06.993452757Z INFO: Collecting host service logs for NetworkManager 2026-09-07T13:27:06.993471621Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-07T13:27:06.993802226Z INFO: Collecting host service logs for openvswitch 2026-09-07T13:27:06.994143593Z INFO: Collecting host service logs for ovs-configuration 2026-09-07T13:27:06.994469854Z INFO: Collecting host service logs for ovsdb-server 2026-09-07T13:27:06.994850223Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-07T13:27:06.995256096Z INFO: Waiting for worker host service log collection to complete ... 2026-09-07T13:27:06.995704789Z Executing Istio gather script 2026-09-07T13:27:06.995916866Z INFO: Gathering HAProxy config files 2026-09-07T13:27:07.276380503Z INFO: Waiting for node performance related collection to complete ... 2026-09-07T13:27:07.330078418Z No resources found in openshift-etcd namespace. 2026-09-07T13:27:07.726435150Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-07T13:27:07.864885713Z Wrote inspect data to must-gather. 2026-09-07T13:27:08.110805039Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-07T13:27:08.148064639Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-07T13:27:08.559351199Z INFO: "metallb-operator" not detected. Skipping. 2026-09-07T13:27:08.597934313Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-07T13:27:08.623781747Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-07T13:27:08.656103608Z No resources found 2026-09-07T13:27:08.756530111Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-07T13:27:08.756627997Z INFO: on-disk MachineConfig config collection complete. 2026-09-07T13:27:08.891868988Z Wrote inspect data to must-gather. 2026-09-07T13:27:08.902141225Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-07T13:27:08.978977595Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T13:27:08.981665258Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-pmg77 2026-09-07T13:27:08.981706987Z insights-runtime-extractor-dgr5h 2026-09-07T13:27:08.981726834Z insights-runtime-extractor-hn7p4 2026-09-07T13:27:08.981746758Z insights-runtime-extractor-rwqd7 2026-09-07T13:27:09.008964705Z error: the server doesn't have a resource type "performanceprofile" 2026-09-07T13:27:09.030225204Z Gathering data for ns/openshift-monitoring... 2026-09-07T13:27:09.061611191Z ERROR: No running kube-apiserver pods found 2026-09-07T13:27:09.159833812Z INFO: Worker host service log collection to complete. 2026-09-07T13:27:09.226073411Z Wrote inspect data to must-gather. 2026-09-07T13:27:09.327246896Z Wrote inspect data to must-gather. 2026-09-07T13:27:09.349197124Z No resources found 2026-09-07T13:27:09.359852013Z tar: Removing leading `/' from member names 2026-09-07T13:27:09.365466425Z INFO: HAProxy config collection complete. 2026-09-07T13:27:09.391800609Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-07T13:27:09.399869680Z Wrote inspect data to must-gather. 2026-09-07T13:27:09.458350026Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-07T13:27:09.458350026Z See 'oc rsync -h' for help and examples 2026-09-07T13:27:09.478763477Z INFO: 'previous-logs' folder not found on ip-10-0-130-185.ec2.internal, skipping... 2026-09-07T13:27:09.496371041Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-07T13:27:09.582958439Z Wrote inspect data to must-gather. 2026-09-07T13:27:09.588380778Z Wrote inspect data to must-gather. 2026-09-07T13:27:09.764133604Z Wrote inspect data to must-gather. 2026-09-07T13:27:09.764746626Z INFO: 'previous-logs' folder not found on ip-10-0-131-110.ec2.internal, skipping... 2026-09-07T13:27:09.777233945Z error: the server doesn't have a resource type "machineconfigs" 2026-09-07T13:27:09.821463432Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-07T13:27:09.957789444Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-07T13:27:10.031442853Z INFO: 'previous-logs' folder not found on ip-10-0-141-217.ec2.internal, skipping... 2026-09-07T13:27:10.031470989Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-07T13:27:10.031502696Z INFO: Machine Config Daemon termination log collection complete. 2026-09-07T13:27:10.032808518Z INFO: Getting rules from prometheus-k8s-0 2026-09-07T13:27:10.044677458Z Wrote inspect data to must-gather. 2026-09-07T13:27:10.179938198Z Wrote inspect data to must-gather. 2026-09-07T13:27:10.297038481Z Wrote inspect data to must-gather. 2026-09-07T13:27:10.336640040Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-07T13:27:10.464889953Z INFO: Getting status/config from prometheus-k8s-0 2026-09-07T13:27:10.550215858Z Wrote inspect data to must-gather. 2026-09-07T13:27:10.593735517Z INFO: INTERCONNECT MODE 2026-09-07T13:27:10.593769456Z INFO: Gathering ovn-kubernetes DBs 2026-09-07T13:27:10.773956487Z INFO: Gathering OVN_Northbound from ovnkube-node-78x7t... 2026-09-07T13:27:10.774361153Z INFO: Gathering OVN_Northbound from ovnkube-node-8grwl... 2026-09-07T13:27:10.774727247Z INFO: Gathering OVN_Northbound from ovnkube-node-l764h... 2026-09-07T13:27:10.775001299Z INFO: Gathering OVN_Southbound from ovnkube-node-78x7t... 2026-09-07T13:27:10.775272236Z INFO: Gathering OVN_Southbound from ovnkube-node-8grwl... 2026-09-07T13:27:10.775549439Z INFO: Gathering OVN_Southbound from ovnkube-node-l764h... 2026-09-07T13:27:10.778172396Z Wrote inspect data to must-gather. 2026-09-07T13:27:11.011051053Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-07T13:27:11.015031267Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-07T13:27:11.038835056Z tar: Removing leading `/' from member names 2026-09-07T13:27:11.116590646Z tar: Removing leading `/' from member names 2026-09-07T13:27:11.167478380Z Wrote inspect data to must-gather. 2026-09-07T13:27:11.238432621Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-07T13:27:11.258583002Z tar: Removing leading `/' from member names 2026-09-07T13:27:11.266693201Z tar: Removing leading `/' from member names 2026-09-07T13:27:11.272186165Z tar: Removing leading `/' from member names 2026-09-07T13:27:11.284918481Z tar: Removing leading `/' from member names 2026-09-07T13:27:11.289033656Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:11.401582967Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-07T13:27:11.441894335Z Wrote inspect data to must-gather. 2026-09-07T13:27:11.546066623Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:11.548379032Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-07T13:27:11.673584376Z INFO: Waiting for network log collection to complete ... 2026-09-07T13:27:11.673619562Z INFO: Waiting for ovnk database copies to complete ... 2026-09-07T13:27:11.674113854Z INFO: Copying ovnk databases complete. 2026-09-07T13:27:11.675718847Z 2.8M must-gather/network_logs/ovnk_database_store 2026-09-07T13:27:11.677939166Z ovnk_database_store/ 2026-09-07T13:27:11.677939166Z ovnk_database_store/ovnkube-node-8grwl_sbdb 2026-09-07T13:27:11.690518514Z Wrote inspect data to must-gather. 2026-09-07T13:27:11.696100554Z ovnk_database_store/ovnkube-node-78x7t_nbdb 2026-09-07T13:27:11.702772354Z ovnk_database_store/ovnkube-node-l764h_nbdb 2026-09-07T13:27:11.711568079Z ovnk_database_store/ovnkube-node-8grwl_nbdb 2026-09-07T13:27:11.719096702Z ovnk_database_store/ovnkube-node-l764h_sbdb 2026-09-07T13:27:11.730544779Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-07T13:27:11.740619654Z ovnk_database_store/ovnkube-node-78x7t_sbdb 2026-09-07T13:27:11.761401482Z INFO: Network log collection complete. 2026-09-07T13:27:11.779411260Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:11.905075612Z W0907 13:27:11.905024 1482 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-07T13:27:11.911485045Z Wrote inspect data to must-gather. 2026-09-07T13:27:11.969598630Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-07T13:27:11.988699594Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:11.991126497Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-07T13:27:12.093455971Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-07T13:27:12.192202965Z Gathering data for ns/openshift-network-console... 2026-09-07T13:27:12.197835892Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:12.227393107Z Wrote inspect data to must-gather. 2026-09-07T13:27:12.407962887Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:12.410790303Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-07T13:27:12.511867202Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-07T13:27:12.514733059Z INFO: Fallback to identify the container image from release info 2026-09-07T13:27:12.528315191Z INFO: Getting status from alertmanager-main-0 2026-09-07T13:27:12.668305428Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:12.755281463Z Gathering data for ns/openshift-console-operator... 2026-09-07T13:27:12.870782392Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:12.874456354Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-07T13:27:13.013736181Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:13.158031585Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399 2026-09-07T13:27:13.234287851Z Wrote inspect data to /must-gather/istio. 2026-09-07T13:27:13.258359420Z Gathering data for ns/openshift-console... 2026-09-07T13:27:13.289145877Z daemonset.apps/perf-node-gather-daemonset created 2026-09-07T13:27:13.411771234Z Waiting for performance profile collector pods to become ready: 1 2026-09-07T13:27:13.931974151Z Done executing Istio gather script 2026-09-07T13:27:13.967553669Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-07T13:27:14.399936691Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-07T13:27:14.532837564Z Waiting for performance profile collector pods to become ready: 2 2026-09-07T13:27:14.534905691Z Gathering data for ns/openshift-dns-operator... 2026-09-07T13:27:14.860100874Z Gathering data for ns/openshift-dns... 2026-09-07T13:27:15.662800682Z Waiting for performance profile collector pods to become ready: 3 2026-09-07T13:27:15.697379246Z Gathering data for ns/openshift-image-registry... 2026-09-07T13:27:16.256134458Z Gathering data for ns/openshift-ingress-operator... 2026-09-07T13:27:16.626871804Z Gathering data for ns/openshift-ingress... 2026-09-07T13:27:16.791546146Z Waiting for performance profile collector pods to become ready: 4 2026-09-07T13:27:17.080194424Z Gathering data for ns/openshift-ingress-canary... 2026-09-07T13:27:17.721284324Z Gathering data for ns/openshift-insights... 2026-09-07T13:27:17.903319389Z Waiting for performance profile collector pods to become ready: 5 2026-09-07T13:27:19.017350571Z Waiting for performance profile collector pods to become ready: 6 2026-09-07T13:27:19.404859396Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-07T13:27:20.129961538Z Waiting for performance profile collector pods to become ready: 7 2026-09-07T13:27:20.277518767Z Gathering data for ns/openshift-jobset-operator... 2026-09-07T13:27:21.177127678Z Gathering data for ns/openshift-config... 2026-09-07T13:27:21.251837046Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-07T13:27:21.484913759Z Collecting performance related data for node ip-10-0-130-185.ec2.internal 2026-09-07T13:27:21.493204110Z Collecting performance related data for node ip-10-0-131-110.ec2.internal 2026-09-07T13:27:21.502224171Z Collecting performance related data for node ip-10-0-141-217.ec2.internal 2026-09-07T13:27:21.568538443Z Gathering data for ns/openshift-config-managed... 2026-09-07T13:27:22.165830940Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-07T13:27:22.549188475Z Gathering data for ns/openshift-kube-apiserver... 2026-09-07T13:27:22.936505995Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-07T13:27:23.316617353Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-07T13:27:23.685839645Z Gathering data for ns/openshift-kube-scheduler... 2026-09-07T13:27:24.136744727Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-07T13:27:24.412243454Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-07T13:27:24.613430390Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-07T13:27:25.133827800Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-07T13:27:25.591904584Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-07T13:27:26.263644729Z Gathering data for ns/openshift-multus... 2026-09-07T13:27:28.065873261Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-07T13:27:29.417323983Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-07T13:27:29.488342154Z Collecting kubelet logs for node ip-10-0-130-185.ec2.internal 2026-09-07T13:27:29.490366208Z Collecting kubelet logs for node ip-10-0-131-110.ec2.internal 2026-09-07T13:27:29.492487834Z Collecting kubelet logs for node ip-10-0-141-217.ec2.internal 2026-09-07T13:27:29.858183656Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-dx8qd namespace 2026-09-07T13:27:29.863748240Z INFO: Node performance data collection complete. 2026-09-07T13:27:29.894028729Z Gathering data for ns/openshift-host-network... 2026-09-07T13:27:30.349484329Z Gathering data for ns/openshift-network-diagnostics... 2026-09-07T13:27:31.008565362Z Gathering data for ns/openshift-network-node-identity... 2026-09-07T13:27:31.463121749Z Gathering data for ns/openshift-network-operator... 2026-09-07T13:27:31.884044726Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-07T13:27:32.238364121Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-07T13:27:32.789679817Z Gathering data for ns/openshift-apiserver-operator... 2026-09-07T13:27:33.144374948Z Gathering data for ns/openshift-apiserver... 2026-09-07T13:27:33.623103797Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-07T13:27:33.977829999Z Gathering data for ns/openshift-controller-manager... 2026-09-07T13:27:34.332935783Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-07T13:27:34.422951013Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70 2026-09-07T13:27:35.030780471Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-07T13:27:35.385230822Z Gathering data for ns/openshift-service-ca-operator... 2026-09-07T13:27:35.724697189Z Gathering data for ns/openshift-service-ca... 2026-09-07T13:27:36.143179764Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-07T13:27:37.132241759Z Wrote inspect data to must-gather. 2026-09-07T13:27:37.132288239Z error: inspection completed with the errors occurred while gathering data: 2026-09-07T13:27:37.132288239Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-07T13:27:37.257552176Z error: the server doesn't have a resource type "clusters" 2026-09-07T13:27:37.391682013Z Caches written to disk