2026-09-17T12:10:39.556394046Z [disk usage checker] Started 2026-09-17T12:10:39.559815252Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T12:10:39.745034718Z Gathering data for ns/openshift-cluster-version... 2026-09-17T12:10:39.868229797Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T12:10:39.957079529Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T12:10:40.049940628Z Gathering data for ns/default... 2026-09-17T12:10:40.389481564Z Gathering data for ns/openshift... 2026-09-17T12:10:40.808533546Z Gathering data for ns/kube-system... 2026-09-17T12:10:41.536779458Z Gathering data for ns/openshift-etcd... 2026-09-17T12:10:41.825096981Z Wrote inspect data to must-gather. 2026-09-17T12:10:41.825152662Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T12:10:41.825152662Z namespaces "assisted-installer" not found 2026-09-17T12:10:42.015382503Z Waiting on subprocesses to finish execution. 2026-09-17T12:10:42.023481401Z INFO: Gathering HAProxy config files 2026-09-17T12:10:42.026632713Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-17T12:10:42.037244903Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-17T12:10:42.043692265Z INFO: Collecting host service logs for crio 2026-09-17T12:10:42.044026921Z INFO: Collecting host service logs for kubelet 2026-09-17T12:10:42.044388851Z INFO: Collecting host service logs for rpm-ostreed 2026-09-17T12:10:42.044733975Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-17T12:10:42.045078417Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-17T12:10:42.045435274Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-17T12:10:42.045770446Z INFO: Collecting host service logs for NetworkManager 2026-09-17T12:10:42.046101592Z INFO: Collecting host service logs for openvswitch 2026-09-17T12:10:42.046452656Z INFO: Collecting host service logs for ovs-configuration 2026-09-17T12:10:42.046830546Z INFO: Collecting host service logs for ovsdb-server 2026-09-17T12:10:42.047177113Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-17T12:10:42.047576997Z INFO: Waiting for worker host service log collection to complete ... 2026-09-17T12:10:42.056610166Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-17T12:10:42.107526857Z Executing Istio gather script 2026-09-17T12:10:42.144989809Z INFO: Waiting for node performance related collection to complete ... 2026-09-17T12:10:42.647737844Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-17T12:10:42.664444788Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-17T12:10:42.810266528Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-17T12:10:42.811354126Z INFO: on-disk MachineConfig config collection complete. 2026-09-17T12:10:42.839496182Z INFO: "metallb-operator" not detected. Skipping. 2026-09-17T12:10:42.843745487Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-17T12:10:43.042836905Z No resources found 2026-09-17T12:10:43.119484352Z INFO: Collecting Insights Archives from insights-operator-5844b856db-skhl2 2026-09-17T12:10:43.119540838Z insights-runtime-extractor-krpbp 2026-09-17T12:10:43.119562836Z insights-runtime-extractor-mcjf5 2026-09-17T12:10:43.119581938Z insights-runtime-extractor-qksp8 2026-09-17T12:10:43.176055082Z error: the server doesn't have a resource type "performanceprofile" 2026-09-17T12:10:43.195705761Z No resources found in openshift-etcd namespace. 2026-09-17T12:10:43.222611176Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-17T12:10:43.224231550Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-17T12:10:43.492493625Z Wrote inspect data to must-gather. 2026-09-17T12:10:43.548470187Z INFO: Worker host service log collection to complete. 2026-09-17T12:10:43.567643115Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-17T12:10:43.947729104Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-17T12:10:43.947729104Z See 'oc rsync -h' for help and examples 2026-09-17T12:10:43.989440732Z tar: Removing leading `/' from member names 2026-09-17T12:10:43.996148019Z INFO: HAProxy config collection complete. 2026-09-17T12:10:44.015030050Z Wrote inspect data to must-gather. 2026-09-17T12:10:44.081356800Z Wrote inspect data to must-gather. 2026-09-17T12:10:44.134825426Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-17T12:10:44.170393222Z No resources found 2026-09-17T12:10:44.232312102Z Wrote inspect data to must-gather. 2026-09-17T12:10:44.267999915Z ERROR: No running kube-apiserver pods found 2026-09-17T12:10:44.276805328Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T12:10:44.315083964Z Wrote inspect data to must-gather. 2026-09-17T12:10:44.346153194Z Wrote inspect data to must-gather. 2026-09-17T12:10:44.449978898Z INFO: 'previous-logs' folder not found on ip-10-0-130-219.ec2.internal, skipping... 2026-09-17T12:10:44.483526792Z Wrote inspect data to must-gather. 2026-09-17T12:10:44.573835821Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T12:10:44.620912995Z INFO: Getting rules from prometheus-k8s-0 2026-09-17T12:10:44.652304436Z Wrote inspect data to must-gather. 2026-09-17T12:10:44.654975264Z Gathering data for ns/openshift-monitoring... 2026-09-17T12:10:44.689527270Z error: the server doesn't have a resource type "machineconfigs" 2026-09-17T12:10:44.760680295Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-17T12:10:44.790775902Z INFO: 'previous-logs' folder not found on ip-10-0-140-171.ec2.internal, skipping... 2026-09-17T12:10:44.877935486Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-17T12:10:44.929782902Z Wrote inspect data to must-gather. 2026-09-17T12:10:45.067181779Z INFO: 'previous-logs' folder not found on ip-10-0-140-196.ec2.internal, skipping... 2026-09-17T12:10:45.067289063Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-17T12:10:45.067342359Z INFO: Machine Config Daemon termination log collection complete. 2026-09-17T12:10:45.067495227Z INFO: Getting status/config from prometheus-k8s-0 2026-09-17T12:10:45.145718949Z Wrote inspect data to must-gather. 2026-09-17T12:10:45.205572957Z Wrote inspect data to must-gather. 2026-09-17T12:10:45.289528218Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-17T12:10:45.466169525Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-17T12:10:45.493743450Z INFO: INTERCONNECT MODE 2026-09-17T12:10:45.493771326Z INFO: Gathering ovn-kubernetes DBs 2026-09-17T12:10:45.498694163Z Wrote inspect data to must-gather. 2026-09-17T12:10:45.607593546Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-17T12:10:45.647985808Z INFO: Gathering OVN_Northbound from ovnkube-node-dtdhb... 2026-09-17T12:10:45.648257435Z INFO: Gathering OVN_Northbound from ovnkube-node-klgtm... 2026-09-17T12:10:45.648497096Z INFO: Gathering OVN_Northbound from ovnkube-node-nn75c... 2026-09-17T12:10:45.649039978Z INFO: Gathering OVN_Southbound from ovnkube-node-dtdhb... 2026-09-17T12:10:45.649258198Z INFO: Gathering OVN_Southbound from ovnkube-node-klgtm... 2026-09-17T12:10:45.649500973Z INFO: Gathering OVN_Southbound from ovnkube-node-nn75c... 2026-09-17T12:10:45.724548629Z Wrote inspect data to must-gather. 2026-09-17T12:10:45.731963095Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-17T12:10:45.873778769Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:45.924335344Z tar: Removing leading `/' from member names 2026-09-17T12:10:45.943842266Z tar: Removing leading `/' from member names 2026-09-17T12:10:45.964159908Z tar: Removing leading `/' from member names 2026-09-17T12:10:45.965797637Z tar: Removing leading `/' from member names 2026-09-17T12:10:45.984255423Z tar: Removing leading `/' from member names 2026-09-17T12:10:45.992309708Z tar: Removing leading `/' from member names 2026-09-17T12:10:46.056675918Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-17T12:10:46.059590039Z Wrote inspect data to must-gather. 2026-09-17T12:10:46.145208007Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:46.147712047Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-17T12:10:46.234868503Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T12:10:46.308987214Z Wrote inspect data to must-gather. 2026-09-17T12:10:46.329762833Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-17T12:10:46.380210953Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:46.507246226Z INFO: Waiting for network log collection to complete ... 2026-09-17T12:10:46.507276145Z INFO: Waiting for ovnk database copies to complete ... 2026-09-17T12:10:46.507661210Z INFO: Copying ovnk databases complete. 2026-09-17T12:10:46.509436584Z 3.6M must-gather/network_logs/ovnk_database_store 2026-09-17T12:10:46.515572661Z ovnk_database_store/ 2026-09-17T12:10:46.515802563Z ovnk_database_store/ovnkube-node-klgtm_nbdb 2026-09-17T12:10:46.532831201Z ovnk_database_store/ovnkube-node-dtdhb_nbdb 2026-09-17T12:10:46.542267997Z ovnk_database_store/ovnkube-node-dtdhb_sbdb 2026-09-17T12:10:46.564022451Z ovnk_database_store/ovnkube-node-klgtm_sbdb 2026-09-17T12:10:46.581820111Z Wrote inspect data to must-gather. 2026-09-17T12:10:46.587471089Z ovnk_database_store/ovnkube-node-nn75c_sbdb 2026-09-17T12:10:46.611789023Z ovnk_database_store/ovnkube-node-nn75c_nbdb 2026-09-17T12:10:46.612424576Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:46.615318809Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-17T12:10:46.624555320Z INFO: Network log collection complete. 2026-09-17T12:10:46.724688884Z INFO: Getting status from alertmanager-main-0 2026-09-17T12:10:46.800541801Z W0917 12:10:46.800489 1546 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-17T12:10:46.805504024Z Wrote inspect data to must-gather. 2026-09-17T12:10:46.805948276Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:46.896034705Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T12:10:47.016779477Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:47.019003399Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-17T12:10:47.070730856Z Gathering data for ns/openshift-network-console... 2026-09-17T12:10:47.112267092Z Wrote inspect data to must-gather. 2026-09-17T12:10:47.243564558Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:47.364359645Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-17T12:10:47.367091348Z INFO: Fallback to identify the container image from release info 2026-09-17T12:10:47.452154825Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:47.454851045Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-17T12:10:47.484612442Z Gathering data for ns/openshift-console-operator... 2026-09-17T12:10:47.616616344Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:47.782812374Z Wrote inspect data to /must-gather/istio. 2026-09-17T12:10:47.836165327Z Gathering data for ns/openshift-console... 2026-09-17T12:10:48.306277676Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-17T12:10:48.389936178Z Done executing Istio gather script 2026-09-17T12:10:48.743313444Z Gathering data for ns/openshift-dns-operator... 2026-09-17T12:10:48.987940862Z Gathering data for ns/openshift-dns... 2026-09-17T12:10:49.502204141Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a 2026-09-17T12:10:49.556899741Z Gathering data for ns/openshift-image-registry... 2026-09-17T12:10:49.579029155Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T12:10:49.620355621Z daemonset.apps/perf-node-gather-daemonset created 2026-09-17T12:10:49.735170325Z Waiting for performance profile collector pods to become ready: 1 2026-09-17T12:10:50.024611692Z Gathering data for ns/openshift-ingress-operator... 2026-09-17T12:10:50.355863332Z Gathering data for ns/openshift-ingress... 2026-09-17T12:10:50.660653504Z Gathering data for ns/openshift-ingress-canary... 2026-09-17T12:10:50.851139015Z Waiting for performance profile collector pods to become ready: 2 2026-09-17T12:10:51.030298912Z Gathering data for ns/openshift-insights... 2026-09-17T12:10:51.961321834Z Waiting for performance profile collector pods to become ready: 3 2026-09-17T12:10:52.932003364Z Gathering data for ns/openshift-config... 2026-09-17T12:10:53.068213649Z Waiting for performance profile collector pods to become ready: 4 2026-09-17T12:10:53.175917992Z Gathering data for ns/openshift-config-managed... 2026-09-17T12:10:53.578859168Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-17T12:10:53.822626737Z Gathering data for ns/openshift-kube-apiserver... 2026-09-17T12:10:54.055624342Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-17T12:10:54.185704597Z Waiting for performance profile collector pods to become ready: 5 2026-09-17T12:10:54.324278825Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-17T12:10:54.580677673Z Gathering data for ns/openshift-kube-scheduler... 2026-09-17T12:10:54.583914954Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T12:10:54.806751606Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-17T12:10:55.067728496Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-17T12:10:55.294622625Z Waiting for performance profile collector pods to become ready: 6 2026-09-17T12:10:55.361199173Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-17T12:10:55.623190783Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-17T12:10:56.051078221Z Gathering data for ns/openshift-multus... 2026-09-17T12:10:56.392225259Z Waiting for performance profile collector pods to become ready: 7 2026-09-17T12:10:57.326527277Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-17T12:10:57.490103613Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-17T12:10:57.705251319Z Collecting performance related data for node ip-10-0-130-219.ec2.internal 2026-09-17T12:10:57.712615477Z Collecting performance related data for node ip-10-0-140-196.ec2.internal 2026-09-17T12:10:57.720388902Z Collecting performance related data for node ip-10-0-140-171.ec2.internal 2026-09-17T12:10:58.743357254Z Gathering data for ns/openshift-host-network... 2026-09-17T12:10:59.086998996Z Gathering data for ns/openshift-network-diagnostics... 2026-09-17T12:10:59.589385400Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T12:10:59.603137349Z Gathering data for ns/openshift-network-node-identity... 2026-09-17T12:11:00.031974294Z Gathering data for ns/openshift-network-operator... 2026-09-17T12:11:00.413783559Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-17T12:11:00.679230390Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-17T12:11:01.128161179Z Gathering data for ns/openshift-apiserver-operator... 2026-09-17T12:11:01.391262338Z Gathering data for ns/openshift-apiserver... 2026-09-17T12:11:01.734018551Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-17T12:11:01.979365259Z Gathering data for ns/openshift-controller-manager... 2026-09-17T12:11:02.238294609Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-17T12:11:02.885345653Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-17T12:11:03.158530577Z Gathering data for ns/openshift-service-ca-operator... 2026-09-17T12:11:03.464644136Z Gathering data for ns/openshift-service-ca... 2026-09-17T12:11:03.819548642Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-17T12:11:04.594397887Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70 2026-09-17T12:11:04.805731023Z Wrote inspect data to must-gather. 2026-09-17T12:11:04.805770363Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T12:11:04.805770363Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-17T12:11:05.675150899Z Collecting kubelet logs for node ip-10-0-130-219.ec2.internal 2026-09-17T12:11:05.677005935Z Collecting kubelet logs for node ip-10-0-140-196.ec2.internal 2026-09-17T12:11:05.679140620Z Collecting kubelet logs for node ip-10-0-140-171.ec2.internal 2026-09-17T12:11:05.994295980Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-8xmpg namespace 2026-09-17T12:11:05.999414349Z INFO: Node performance data collection complete. 2026-09-17T12:11:06.125282436Z error: the server doesn't have a resource type "clusters" 2026-09-17T12:11:06.277082495Z Caches written to disk