2026-09-16T17:55:52.148486194Z [disk usage checker] Started 2026-09-16T17:55:52.151920741Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:55:52.368129216Z Gathering data for ns/openshift-cluster-version... 2026-09-16T17:55:52.537422355Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T17:55:52.664755409Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T17:55:52.760356065Z Gathering data for ns/default... 2026-09-16T17:55:53.157036333Z Gathering data for ns/openshift... 2026-09-16T17:55:53.685281477Z Gathering data for ns/kube-system... 2026-09-16T17:55:54.644991942Z Gathering data for ns/openshift-etcd... 2026-09-16T17:55:55.071279641Z Waiting on subprocesses to finish execution. 2026-09-16T17:55:55.072717405Z Wrote inspect data to must-gather. 2026-09-16T17:55:55.072804138Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T17:55:55.072804138Z namespaces "assisted-installer" not found 2026-09-16T17:55:55.079753395Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-16T17:55:55.082094076Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-16T17:55:55.086021197Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-16T17:55:55.088259191Z INFO: Gathering HAProxy config files 2026-09-16T17:55:55.089563475Z Executing Istio gather script 2026-09-16T17:55:55.101860059Z INFO: Collecting host service logs for crio 2026-09-16T17:55:55.102213020Z INFO: Collecting host service logs for kubelet 2026-09-16T17:55:55.102568213Z INFO: Collecting host service logs for rpm-ostreed 2026-09-16T17:55:55.102933239Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-16T17:55:55.103268696Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-16T17:55:55.103608393Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-16T17:55:55.103958717Z INFO: Collecting host service logs for NetworkManager 2026-09-16T17:55:55.104299124Z INFO: Collecting host service logs for openvswitch 2026-09-16T17:55:55.104630145Z INFO: Collecting host service logs for ovs-configuration 2026-09-16T17:55:55.105061641Z INFO: Collecting host service logs for ovsdb-server 2026-09-16T17:55:55.105444591Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-16T17:55:55.105878003Z INFO: Waiting for worker host service log collection to complete ... 2026-09-16T17:55:55.214096123Z INFO: Waiting for node performance related collection to complete ... 2026-09-16T17:55:55.671900216Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-16T17:55:55.831038235Z No resources found in openshift-etcd namespace. 2026-09-16T17:55:55.860156157Z INFO: Collecting Insights Archives from insights-operator-5844b856db-gghkt 2026-09-16T17:55:55.860202637Z insights-runtime-extractor-pmhdq 2026-09-16T17:55:55.860222665Z insights-runtime-extractor-stbrb 2026-09-16T17:55:55.860241906Z insights-runtime-extractor-zhxvb 2026-09-16T17:55:55.904428735Z Wrote inspect data to must-gather. 2026-09-16T17:55:56.001703892Z INFO: "metallb-operator" not detected. Skipping. 2026-09-16T17:55:56.051388953Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-16T17:55:56.057275679Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-16T17:55:56.210003589Z No resources found 2026-09-16T17:55:56.405014915Z ERROR: No running kube-apiserver pods found 2026-09-16T17:55:56.515669801Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-16T17:55:56.516347767Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-16T17:55:56.516373868Z INFO: on-disk MachineConfig config collection complete. 2026-09-16T17:55:56.599645606Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-16T17:55:56.599645606Z See 'oc rsync -h' for help and examples 2026-09-16T17:55:56.616886595Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-16T17:55:56.755065068Z error: the server doesn't have a resource type "performanceprofile" 2026-09-16T17:55:56.761504326Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-16T17:55:56.978500142Z INFO: Worker host service log collection to complete. 2026-09-16T17:55:57.128082259Z tar: Removing leading `/' from member names 2026-09-16T17:55:57.134561429Z INFO: HAProxy config collection complete. 2026-09-16T17:55:57.192312163Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:55:57.423076976Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-16T17:55:57.477369034Z No resources found 2026-09-16T17:55:57.535710975Z Wrote inspect data to must-gather. 2026-09-16T17:55:57.675583719Z Wrote inspect data to must-gather. 2026-09-16T17:55:57.690268742Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T17:55:57.720982936Z Wrote inspect data to must-gather. 2026-09-16T17:55:57.738789403Z Wrote inspect data to must-gather. 2026-09-16T17:55:57.771030985Z Wrote inspect data to must-gather. 2026-09-16T17:55:57.800698463Z Gathering data for ns/openshift-monitoring... 2026-09-16T17:55:57.822229578Z INFO: 'previous-logs' folder not found on ip-10-0-129-160.ec2.internal, skipping... 2026-09-16T17:55:57.847549438Z Wrote inspect data to must-gather. 2026-09-16T17:55:58.061010026Z INFO: Getting rules from prometheus-k8s-0 2026-09-16T17:55:58.082038312Z error: the server doesn't have a resource type "machineconfigs" 2026-09-16T17:55:58.206271308Z INFO: 'previous-logs' folder not found on ip-10-0-134-96.ec2.internal, skipping... 2026-09-16T17:55:58.217140283Z Wrote inspect data to must-gather. 2026-09-16T17:55:58.283245562Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-16T17:55:58.521443963Z INFO: 'previous-logs' folder not found on ip-10-0-138-73.ec2.internal, skipping... 2026-09-16T17:55:58.521476138Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-16T17:55:58.521515928Z INFO: Machine Config Daemon termination log collection complete. 2026-09-16T17:55:58.522249174Z Wrote inspect data to must-gather. 2026-09-16T17:55:58.541265495Z Wrote inspect data to must-gather. 2026-09-16T17:55:58.633964115Z INFO: Getting status/config from prometheus-k8s-0 2026-09-16T17:55:58.637654592Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-16T17:55:58.727717742Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-16T17:55:58.792720974Z Wrote inspect data to must-gather. 2026-09-16T17:55:58.957413482Z Wrote inspect data to must-gather. 2026-09-16T17:55:59.109600113Z INFO: INTERCONNECT MODE 2026-09-16T17:55:59.109688213Z INFO: Gathering ovn-kubernetes DBs 2026-09-16T17:55:59.124650276Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-16T17:55:59.208295636Z Wrote inspect data to must-gather. 2026-09-16T17:55:59.293203110Z INFO: Gathering OVN_Northbound from ovnkube-node-f9qxl... 2026-09-16T17:55:59.293470875Z INFO: Gathering OVN_Northbound from ovnkube-node-mrjcs... 2026-09-16T17:55:59.293703135Z INFO: Gathering OVN_Northbound from ovnkube-node-t6p26... 2026-09-16T17:55:59.294015563Z INFO: Gathering OVN_Southbound from ovnkube-node-f9qxl... 2026-09-16T17:55:59.294252248Z INFO: Gathering OVN_Southbound from ovnkube-node-mrjcs... 2026-09-16T17:55:59.294544472Z INFO: Gathering OVN_Southbound from ovnkube-node-t6p26... 2026-09-16T17:55:59.465513818Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-16T17:55:59.535147660Z Wrote inspect data to must-gather. 2026-09-16T17:55:59.653460713Z tar: Removing leading `/' from member names 2026-09-16T17:55:59.671534861Z tar: Removing leading `/' from member names 2026-09-16T17:55:59.692118506Z tar: Removing leading `/' from member names 2026-09-16T17:55:59.720001858Z tar: Removing leading `/' from member names 2026-09-16T17:55:59.720292243Z tar: Removing leading `/' from member names 2026-09-16T17:55:59.720757710Z tar: Removing leading `/' from member names 2026-09-16T17:55:59.790486399Z Wrote inspect data to must-gather. 2026-09-16T17:55:59.909082583Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-16T17:56:00.056285207Z Wrote inspect data to must-gather. 2026-09-16T17:56:00.143352036Z INFO: Waiting for network log collection to complete ... 2026-09-16T17:56:00.143393686Z INFO: Waiting for ovnk database copies to complete ... 2026-09-16T17:56:00.143837151Z INFO: Copying ovnk databases complete. 2026-09-16T17:56:00.146360195Z 3.7M must-gather/network_logs/ovnk_database_store 2026-09-16T17:56:00.148350638Z ovnk_database_store/ 2026-09-16T17:56:00.148350638Z ovnk_database_store/ovnkube-node-t6p26_nbdb 2026-09-16T17:56:00.157925784Z ovnk_database_store/ovnkube-node-mrjcs_nbdb 2026-09-16T17:56:00.159965679Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-16T17:56:00.167475663Z ovnk_database_store/ovnkube-node-t6p26_sbdb 2026-09-16T17:56:00.192014140Z ovnk_database_store/ovnkube-node-mrjcs_sbdb 2026-09-16T17:56:00.216384806Z ovnk_database_store/ovnkube-node-f9qxl_nbdb 2026-09-16T17:56:00.227765267Z ovnk_database_store/ovnkube-node-f9qxl_sbdb 2026-09-16T17:56:00.258501780Z INFO: Network log collection complete. 2026-09-16T17:56:00.285874335Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-16T17:56:00.303623741Z Wrote inspect data to must-gather. 2026-09-16T17:56:00.373334015Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:00.394556389Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T17:56:00.506177572Z Wrote inspect data to must-gather. 2026-09-16T17:56:00.599143461Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:00.601643397Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-16T17:56:00.794381780Z INFO: Getting status from alertmanager-main-0 2026-09-16T17:56:00.823562502Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:00.825817228Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-16T17:56:00.828742645Z INFO: Fallback to identify the container image from release info 2026-09-16T17:56:01.048117756Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:01.050383667Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-16T17:56:01.082584341Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T17:56:01.237235656Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:01.265017005Z Gathering data for ns/openshift-network-console... 2026-09-16T17:56:01.411563928Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-16T17:56:01.446715038Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:01.449429588Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-16T17:56:01.547220641Z daemonset.apps/perf-node-gather-daemonset created 2026-09-16T17:56:01.695734549Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:01.709003492Z Waiting for performance profile collector pods to become ready: 1 2026-09-16T17:56:01.818106817Z Gathering data for ns/openshift-console-operator... 2026-09-16T17:56:01.917552818Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:01.920721312Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-16T17:56:02.098511414Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:02.215626743Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:56:02.266116617Z Gathering data for ns/openshift-console... 2026-09-16T17:56:02.305509436Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:02.307552500Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-16T17:56:02.479838490Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:02.683103916Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:02.686309251Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-16T17:56:02.846265665Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-16T17:56:02.864307626Z Waiting for performance profile collector pods to become ready: 2 2026-09-16T17:56:02.876112652Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:03.075634562Z Wrote inspect data to /must-gather/istio. 2026-09-16T17:56:03.442046063Z Gathering data for ns/openshift-dns-operator... 2026-09-16T17:56:03.813670461Z Done executing Istio gather script 2026-09-16T17:56:03.829508322Z Gathering data for ns/openshift-dns... 2026-09-16T17:56:03.994802017Z Waiting for performance profile collector pods to become ready: 3 2026-09-16T17:56:04.781360638Z Gathering data for ns/openshift-image-registry... 2026-09-16T17:56:05.130102118Z Waiting for performance profile collector pods to become ready: 4 2026-09-16T17:56:05.489683865Z Gathering data for ns/openshift-ingress-operator... 2026-09-16T17:56:06.057272065Z Gathering data for ns/openshift-ingress... 2026-09-16T17:56:06.263698287Z Waiting for performance profile collector pods to become ready: 5 2026-09-16T17:56:06.478137579Z Gathering data for ns/openshift-ingress-canary... 2026-09-16T17:56:06.949683086Z Gathering data for ns/openshift-insights... 2026-09-16T17:56:07.220104391Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:56:07.399474096Z Waiting for performance profile collector pods to become ready: 6 2026-09-16T17:56:08.520157426Z Waiting for performance profile collector pods to become ready: 7 2026-09-16T17:56:09.536456266Z Gathering data for ns/kserve... 2026-09-16T17:56:09.647618558Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-16T17:56:09.912697186Z Collecting performance related data for node ip-10-0-138-73.ec2.internal 2026-09-16T17:56:09.919397987Z Collecting performance related data for node ip-10-0-129-160.ec2.internal 2026-09-16T17:56:09.926927374Z Collecting performance related data for node ip-10-0-134-96.ec2.internal 2026-09-16T17:56:11.688921033Z Gathering data for ns/openshift-config... 2026-09-16T17:56:12.086784977Z Gathering data for ns/openshift-config-managed... 2026-09-16T17:56:12.224619954Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:56:12.591194505Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-16T17:56:12.975003081Z Gathering data for ns/openshift-kube-apiserver... 2026-09-16T17:56:13.347033448Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-16T17:56:13.675667148Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-16T17:56:14.046509166Z Gathering data for ns/openshift-kube-scheduler... 2026-09-16T17:56:14.363129446Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-16T17:56:14.718816949Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-16T17:56:15.301775728Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-16T17:56:15.758233643Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-16T17:56:16.467760197Z Gathering data for ns/openshift-multus... 2026-09-16T17:56:17.229071516Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:56:18.502738039Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-16T17:56:18.676786607Z Collecting kubelet logs for node ip-10-0-138-73.ec2.internal 2026-09-16T17:56:18.678570765Z Collecting kubelet logs for node ip-10-0-129-160.ec2.internal 2026-09-16T17:56:18.680359035Z Collecting kubelet logs for node ip-10-0-134-96.ec2.internal 2026-09-16T17:56:19.056177705Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-znfk5 namespace 2026-09-16T17:56:19.065088758Z INFO: Node performance data collection complete. 2026-09-16T17:56:20.112780935Z Gathering data for ns/openshift-host-network... 2026-09-16T17:56:20.435577412Z Gathering data for ns/openshift-network-diagnostics... 2026-09-16T17:56:21.110367819Z Gathering data for ns/openshift-network-node-identity... 2026-09-16T17:56:21.617632015Z Gathering data for ns/openshift-network-operator... 2026-09-16T17:56:22.204324006Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-16T17:56:22.234799907Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:56:22.585649185Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-16T17:56:23.316455804Z Gathering data for ns/openshift-apiserver-operator... 2026-09-16T17:56:23.813529441Z Gathering data for ns/openshift-apiserver... 2026-09-16T17:56:24.312160667Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-16T17:56:24.641676865Z Gathering data for ns/openshift-controller-manager... 2026-09-16T17:56:25.042287842Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-16T17:56:25.836347145Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-16T17:56:26.315673445Z Gathering data for ns/openshift-service-ca-operator... 2026-09-16T17:56:26.814213632Z Gathering data for ns/openshift-service-ca... 2026-09-16T17:56:27.240263019Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70 2026-09-16T17:56:27.348828308Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-16T17:56:28.643341480Z Wrote inspect data to must-gather. 2026-09-16T17:56:28.643389898Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T17:56:28.643389898Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-16T17:56:28.773910524Z error: the server doesn't have a resource type "clusters" 2026-09-16T17:56:28.842912229Z Caches written to disk