2026-09-16T19:13:58.972633577Z [disk usage checker] Started 2026-09-16T19:13:58.976239268Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:13:59.207152219Z Gathering data for ns/openshift-cluster-version... 2026-09-16T19:13:59.419306428Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T19:13:59.578644516Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T19:13:59.713373891Z Gathering data for ns/default... 2026-09-16T19:14:00.156727315Z Gathering data for ns/openshift... 2026-09-16T19:14:00.781842130Z Gathering data for ns/kube-system... 2026-09-16T19:14:01.876471655Z Waiting on subprocesses to finish execution. 2026-09-16T19:14:01.882655859Z INFO: Collecting host service logs for crio 2026-09-16T19:14:01.882979495Z INFO: Collecting host service logs for kubelet 2026-09-16T19:14:01.883160450Z INFO: Gathering HAProxy config files 2026-09-16T19:14:01.883324473Z INFO: Collecting host service logs for rpm-ostreed 2026-09-16T19:14:01.883626615Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-16T19:14:01.883929673Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-16T19:14:01.884685067Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-16T19:14:01.884685067Z INFO: Collecting host service logs for NetworkManager 2026-09-16T19:14:01.884837053Z INFO: Collecting host service logs for openvswitch 2026-09-16T19:14:01.885145764Z INFO: Collecting host service logs for ovs-configuration 2026-09-16T19:14:01.885453234Z INFO: Collecting host service logs for ovsdb-server 2026-09-16T19:14:01.885749591Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-16T19:14:01.886106985Z INFO: Waiting for worker host service log collection to complete ... 2026-09-16T19:14:01.894816803Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-16T19:14:01.896347939Z Executing Istio gather script 2026-09-16T19:14:01.937136459Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-16T19:14:01.938446857Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-16T19:14:02.018752607Z INFO: Waiting for node performance related collection to complete ... 2026-09-16T19:14:02.129442147Z Gathering data for ns/openshift-etcd... 2026-09-16T19:14:02.650519376Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-16T19:14:02.900052017Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-16T19:14:03.051111076Z INFO: Collecting Insights Archives from insights-operator-5844b856db-kvg64 2026-09-16T19:14:03.051111076Z insights-runtime-extractor-67b59 2026-09-16T19:14:03.051111076Z insights-runtime-extractor-mzslw 2026-09-16T19:14:03.051111076Z insights-runtime-extractor-nnb94 2026-09-16T19:14:03.093704643Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-16T19:14:03.093788744Z INFO: on-disk MachineConfig config collection complete. 2026-09-16T19:14:03.128556923Z No resources found in openshift-etcd namespace. 2026-09-16T19:14:03.179634703Z error: the server doesn't have a resource type "performanceprofile" 2026-09-16T19:14:03.233681532Z No resources found 2026-09-16T19:14:03.345429393Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-16T19:14:03.747058409Z Wrote inspect data to must-gather. 2026-09-16T19:14:03.889677021Z Wrote inspect data to must-gather. 2026-09-16T19:14:03.893273484Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-16T19:14:03.954364875Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T19:14:03.968094154Z No resources found 2026-09-16T19:14:03.997098610Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-16T19:14:04.065293368Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:04.090855970Z Wrote inspect data to must-gather. 2026-09-16T19:14:04.091006690Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T19:14:04.091006690Z namespaces "assisted-installer" not found 2026-09-16T19:14:04.093443272Z INFO: "metallb-operator" not detected. Skipping. 2026-09-16T19:14:04.199008111Z INFO: Worker host service log collection to complete. 2026-09-16T19:14:04.265346449Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-16T19:14:04.265346449Z See 'oc rsync -h' for help and examples 2026-09-16T19:14:04.337406552Z Wrote inspect data to must-gather. 2026-09-16T19:14:04.351530102Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-16T19:14:04.353696292Z Wrote inspect data to must-gather. 2026-09-16T19:14:04.511028082Z Wrote inspect data to must-gather. 2026-09-16T19:14:04.599806506Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-16T19:14:04.659317863Z Wrote inspect data to must-gather. 2026-09-16T19:14:04.659467116Z ERROR: No running kube-apiserver pods found 2026-09-16T19:14:04.782327324Z Wrote inspect data to must-gather. 2026-09-16T19:14:04.812035282Z error: the server doesn't have a resource type "machineconfigs" 2026-09-16T19:14:04.845106485Z INFO: 'previous-logs' folder not found on ip-10-0-135-229.ec2.internal, skipping... 2026-09-16T19:14:04.871375569Z Gathering data for ns/openshift-monitoring... 2026-09-16T19:14:04.884506665Z tar: Removing leading `/' from member names 2026-09-16T19:14:04.890317651Z INFO: HAProxy config collection complete. 2026-09-16T19:14:04.914492941Z Wrote inspect data to must-gather. 2026-09-16T19:14:05.024707472Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-16T19:14:05.170204942Z INFO: 'previous-logs' folder not found on ip-10-0-135-236.ec2.internal, skipping... 2026-09-16T19:14:05.197470545Z INFO: Getting rules from prometheus-k8s-0 2026-09-16T19:14:05.249419746Z Wrote inspect data to must-gather. 2026-09-16T19:14:05.291513455Z Wrote inspect data to must-gather. 2026-09-16T19:14:05.480373910Z INFO: 'previous-logs' folder not found on ip-10-0-137-173.ec2.internal, skipping... 2026-09-16T19:14:05.480404496Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-16T19:14:05.480446775Z INFO: Machine Config Daemon termination log collection complete. 2026-09-16T19:14:05.485461239Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-16T19:14:05.529820024Z Wrote inspect data to must-gather. 2026-09-16T19:14:05.652010337Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-16T19:14:05.725772516Z INFO: Getting status/config from prometheus-k8s-0 2026-09-16T19:14:05.735203406Z Wrote inspect data to must-gather. 2026-09-16T19:14:05.860686816Z INFO: INTERCONNECT MODE 2026-09-16T19:14:05.860718540Z INFO: Gathering ovn-kubernetes DBs 2026-09-16T19:14:05.950972115Z Wrote inspect data to must-gather. 2026-09-16T19:14:06.039895597Z INFO: Gathering OVN_Northbound from ovnkube-node-7fgzh... 2026-09-16T19:14:06.040124166Z INFO: Gathering OVN_Northbound from ovnkube-node-l62v6... 2026-09-16T19:14:06.040370668Z INFO: Gathering OVN_Northbound from ovnkube-node-xqn46... 2026-09-16T19:14:06.040637933Z INFO: Gathering OVN_Southbound from ovnkube-node-7fgzh... 2026-09-16T19:14:06.040901106Z INFO: Gathering OVN_Southbound from ovnkube-node-l62v6... 2026-09-16T19:14:06.041215231Z INFO: Gathering OVN_Southbound from ovnkube-node-xqn46... 2026-09-16T19:14:06.298014560Z Wrote inspect data to must-gather. 2026-09-16T19:14:06.307444262Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-16T19:14:06.412713924Z tar: Removing leading `/' from member names 2026-09-16T19:14:06.421206196Z tar: Removing leading `/' from member names 2026-09-16T19:14:06.427189471Z tar: Removing leading `/' from member names 2026-09-16T19:14:06.450861287Z tar: Removing leading `/' from member names 2026-09-16T19:14:06.461809711Z tar: Removing leading `/' from member names 2026-09-16T19:14:06.470875182Z tar: Removing leading `/' from member names 2026-09-16T19:14:06.555285309Z Wrote inspect data to must-gather. 2026-09-16T19:14:06.789575139Z Wrote inspect data to must-gather. 2026-09-16T19:14:06.815881272Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-16T19:14:06.928766960Z INFO: Waiting for network log collection to complete ... 2026-09-16T19:14:06.928806362Z INFO: Waiting for ovnk database copies to complete ... 2026-09-16T19:14:06.929345003Z INFO: Copying ovnk databases complete. 2026-09-16T19:14:06.931562262Z 2.8M must-gather/network_logs/ovnk_database_store 2026-09-16T19:14:06.933895424Z ovnk_database_store/ 2026-09-16T19:14:06.933895424Z ovnk_database_store/ovnkube-node-l62v6_nbdb 2026-09-16T19:14:06.940626134Z ovnk_database_store/ovnkube-node-xqn46_nbdb 2026-09-16T19:14:06.947255072Z ovnk_database_store/ovnkube-node-l62v6_sbdb 2026-09-16T19:14:06.965469124Z ovnk_database_store/ovnkube-node-7fgzh_sbdb 2026-09-16T19:14:06.980458588Z ovnk_database_store/ovnkube-node-7fgzh_nbdb 2026-09-16T19:14:06.985882120Z ovnk_database_store/ovnkube-node-xqn46_sbdb 2026-09-16T19:14:07.006088411Z INFO: Network log collection complete. 2026-09-16T19:14:07.058333177Z Wrote inspect data to must-gather. 2026-09-16T19:14:07.068968784Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-16T19:14:07.115475173Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-16T19:14:07.253977869Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:07.273267535Z Wrote inspect data to must-gather. 2026-09-16T19:14:07.472106710Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:07.474544832Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-16T19:14:07.480601229Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-16T19:14:07.547263276Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-16T19:14:07.549464201Z INFO: Fallback to identify the container image from release info 2026-09-16T19:14:07.686019473Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:07.845315214Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-16T19:14:07.884382670Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:07.886531061Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-16T19:14:07.917921733Z INFO: Getting status from alertmanager-main-0 2026-09-16T19:14:08.101475112Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:08.317514869Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:08.320862438Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-16T19:14:08.334922042Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-16T19:14:08.467228332Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-16T19:14:08.481413356Z daemonset.apps/perf-node-gather-daemonset created 2026-09-16T19:14:08.570372667Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:08.642654686Z Waiting for performance profile collector pods to become ready: 1 2026-09-16T19:14:08.733268768Z Gathering data for ns/openshift-network-console... 2026-09-16T19:14:08.795403797Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:08.797757385Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-16T19:14:08.988768270Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:09.090223840Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:09.217688234Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:09.221355312Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-16T19:14:09.344042724Z Gathering data for ns/openshift-console-operator... 2026-09-16T19:14:09.401090757Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:09.601332279Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:09.603505967Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-16T19:14:09.801776815Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:09.802851170Z Waiting for performance profile collector pods to become ready: 2 2026-09-16T19:14:09.967926850Z Gathering data for ns/openshift-console... 2026-09-16T19:14:09.997985997Z Wrote inspect data to /must-gather/istio. 2026-09-16T19:14:10.769728855Z Done executing Istio gather script 2026-09-16T19:14:10.803203701Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-16T19:14:10.928578208Z Waiting for performance profile collector pods to become ready: 3 2026-09-16T19:14:11.454153970Z Gathering data for ns/openshift-dns-operator... 2026-09-16T19:14:11.837992534Z Gathering data for ns/openshift-dns... 2026-09-16T19:14:12.045954390Z Waiting for performance profile collector pods to become ready: 4 2026-09-16T19:14:12.873841721Z Gathering data for ns/openshift-image-registry... 2026-09-16T19:14:13.161556995Z Waiting for performance profile collector pods to become ready: 5 2026-09-16T19:14:13.668333244Z Gathering data for ns/openshift-ingress-operator... 2026-09-16T19:14:14.095664311Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:14.136217220Z Gathering data for ns/openshift-ingress... 2026-09-16T19:14:14.310235948Z Waiting for performance profile collector pods to become ready: 6 2026-09-16T19:14:14.683421188Z Gathering data for ns/openshift-ingress-canary... 2026-09-16T19:14:15.369809145Z Gathering data for ns/openshift-insights... 2026-09-16T19:14:15.435878371Z Waiting for performance profile collector pods to become ready: 7 2026-09-16T19:14:16.574841702Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-16T19:14:16.846894318Z Collecting performance related data for node ip-10-0-137-173.ec2.internal 2026-09-16T19:14:16.853315211Z Collecting performance related data for node ip-10-0-135-236.ec2.internal 2026-09-16T19:14:16.860427017Z Collecting performance related data for node ip-10-0-135-229.ec2.internal 2026-09-16T19:14:19.082469644Z Gathering data for ns/kserve... 2026-09-16T19:14:19.103189224Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:20.894802436Z Gathering data for ns/openshift-config... 2026-09-16T19:14:21.356480168Z Gathering data for ns/openshift-config-managed... 2026-09-16T19:14:21.890006341Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-16T19:14:22.302756111Z Gathering data for ns/openshift-kube-apiserver... 2026-09-16T19:14:22.743747571Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-16T19:14:23.340541764Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-16T19:14:23.846206313Z Gathering data for ns/openshift-kube-scheduler... 2026-09-16T19:14:24.108095132Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:24.304167298Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-16T19:14:24.746973201Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-16T19:14:25.205975166Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-16T19:14:25.688295018Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-16T19:14:25.944735401Z Collecting kubelet logs for node ip-10-0-137-173.ec2.internal 2026-09-16T19:14:25.946905938Z Collecting kubelet logs for node ip-10-0-135-236.ec2.internal 2026-09-16T19:14:25.949037698Z Collecting kubelet logs for node ip-10-0-135-229.ec2.internal 2026-09-16T19:14:26.328225881Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-tchx8 namespace 2026-09-16T19:14:26.337635555Z INFO: Node performance data collection complete. 2026-09-16T19:14:26.498007635Z Gathering data for ns/openshift-multus... 2026-09-16T19:14:29.112858757Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:29.268582626Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-16T19:14:31.757992030Z Gathering data for ns/openshift-host-network... 2026-09-16T19:14:32.280657312Z Gathering data for ns/openshift-network-diagnostics... 2026-09-16T19:14:33.205689239Z Gathering data for ns/openshift-network-node-identity... 2026-09-16T19:14:33.835573486Z Gathering data for ns/openshift-network-operator... 2026-09-16T19:14:34.117318171Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:34.491803407Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-16T19:14:34.928097521Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-16T19:14:35.739172484Z Gathering data for ns/openshift-apiserver-operator... 2026-09-16T19:14:36.222949664Z Gathering data for ns/openshift-apiserver... 2026-09-16T19:14:36.822655422Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-16T19:14:37.287075541Z Gathering data for ns/openshift-controller-manager... 2026-09-16T19:14:37.696161732Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-16T19:14:38.540254054Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-16T19:14:39.027675373Z Gathering data for ns/openshift-service-ca-operator... 2026-09-16T19:14:39.124356010Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-16T19:14:39.563856445Z Gathering data for ns/openshift-service-ca... 2026-09-16T19:14:40.122210840Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-16T19:14:41.363519445Z Wrote inspect data to must-gather. 2026-09-16T19:14:41.363569215Z error: inspection completed with the errors occurred while gathering data: 2026-09-16T19:14:41.363569215Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-16T19:14:41.504040062Z error: the server doesn't have a resource type "clusters" 2026-09-16T19:14:41.635958407Z Caches written to disk