2026-09-17T16:24:31.699474707Z [disk usage checker] Started
2026-09-17T16:24:31.703328470Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-09-17T16:24:31.908049116Z Gathering data for ns/openshift-cluster-version...
2026-09-17T16:24:32.061484113Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T16:24:32.178841445Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:24:32.269380380Z Gathering data for ns/default...
2026-09-17T16:24:32.595541894Z Gathering data for ns/openshift...
2026-09-17T16:24:33.026560925Z Gathering data for ns/kube-system...
2026-09-17T16:24:33.768453759Z Gathering data for ns/openshift-etcd...
2026-09-17T16:24:34.091732688Z Wrote inspect data to must-gather.
2026-09-17T16:24:34.091847205Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T16:24:34.091847205Z     namespaces "assisted-installer" not found
2026-09-17T16:24:34.500814823Z Waiting on subprocesses to finish execution.
2026-09-17T16:24:34.509146079Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T16:24:34.510622495Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T16:24:34.511041022Z Executing Istio gather script
2026-09-17T16:24:34.515456337Z INFO: Collecting host service logs for crio
2026-09-17T16:24:34.515734237Z INFO: Collecting host service logs for kubelet
2026-09-17T16:24:34.516033024Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T16:24:34.516345481Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T16:24:34.516725241Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T16:24:34.517086950Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T16:24:34.517440831Z INFO: Collecting host service logs for NetworkManager
2026-09-17T16:24:34.517763682Z INFO: Collecting host service logs for openvswitch
2026-09-17T16:24:34.518102503Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T16:24:34.518427881Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T16:24:34.518758701Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T16:24:34.519171631Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T16:24:34.529918747Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T16:24:34.533883466Z INFO: Gathering HAProxy config files
2026-09-17T16:24:34.553016761Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T16:24:35.080515375Z No resources found
2026-09-17T16:24:35.609712480Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T16:24:35.676360932Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T16:24:35.695347126Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T16:24:35.774655221Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T16:24:35.936170373Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-btkfl
2026-09-17T16:24:35.936217103Z insights-runtime-extractor-72tfl
2026-09-17T16:24:35.936237469Z insights-runtime-extractor-jclm2
2026-09-17T16:24:35.936257201Z insights-runtime-extractor-ql9hb
2026-09-17T16:24:35.948553076Z No resources found in openshift-etcd namespace.
2026-09-17T16:24:35.986662804Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T16:24:35.998197034Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T16:24:36.100760275Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T16:24:36.159052659Z INFO: Worker host service log collection to complete.
2026-09-17T16:24:36.198966142Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T16:24:36.199050068Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T16:24:36.418980518Z Wrote inspect data to must-gather.
2026-09-17T16:24:36.533728194Z Wrote inspect data to must-gather.
2026-09-17T16:24:36.596864008Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T16:24:36.596864008Z See 'oc rsync -h' for help and examples
2026-09-17T16:24:36.603910670Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:24:36.641689862Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T16:24:36.698404826Z Gathering data for ns/openshift-monitoring...
2026-09-17T16:24:36.757935724Z tar: Removing leading `/' from member names
2026-09-17T16:24:36.763622345Z INFO: HAProxy config collection complete.
2026-09-17T16:24:36.802823455Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-09-17T16:24:36.827428824Z ERROR: No running kube-apiserver pods found
2026-09-17T16:24:36.855646294Z Wrote inspect data to must-gather.
2026-09-17T16:24:36.928596823Z No resources found
2026-09-17T16:24:36.937415140Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T16:24:36.964759415Z INFO: 'previous-logs' folder not found on ip-10-0-128-128.ec2.internal, skipping...
2026-09-17T16:24:37.120292997Z Wrote inspect data to must-gather.
2026-09-17T16:24:37.139363838Z Wrote inspect data to must-gather.
2026-09-17T16:24:37.208263327Z Wrote inspect data to must-gather.
2026-09-17T16:24:37.229664571Z Wrote inspect data to must-gather.
2026-09-17T16:24:37.307968442Z Wrote inspect data to must-gather.
2026-09-17T16:24:37.356407476Z INFO: 'previous-logs' folder not found on ip-10-0-128-191.ec2.internal, skipping...
2026-09-17T16:24:37.416101960Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T16:24:37.446181904Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T16:24:37.630805934Z Wrote inspect data to must-gather.
2026-09-17T16:24:37.645524955Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T16:24:37.691663403Z INFO: 'previous-logs' folder not found on ip-10-0-131-144.ec2.internal, skipping...
2026-09-17T16:24:37.691700455Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T16:24:37.691700455Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T16:24:37.888424845Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T16:24:37.890024706Z Wrote inspect data to must-gather.
2026-09-17T16:24:37.897613024Z Wrote inspect data to must-gather.
2026-09-17T16:24:38.072286996Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T16:24:38.226855269Z INFO: INTERCONNECT MODE
2026-09-17T16:24:38.226978810Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T16:24:38.244691179Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T16:24:38.322885427Z Wrote inspect data to must-gather.
2026-09-17T16:24:38.375734406Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T16:24:38.396196699Z INFO: Gathering OVN_Northbound from ovnkube-node-h25hc...
2026-09-17T16:24:38.396451925Z INFO: Gathering OVN_Northbound from ovnkube-node-szc22...
2026-09-17T16:24:38.396679954Z INFO: Gathering OVN_Northbound from ovnkube-node-z8nmw...
2026-09-17T16:24:38.396980675Z INFO: Gathering OVN_Southbound from ovnkube-node-h25hc...
2026-09-17T16:24:38.397253157Z INFO: Gathering OVN_Southbound from ovnkube-node-szc22...
2026-09-17T16:24:38.397545729Z INFO: Gathering OVN_Southbound from ovnkube-node-z8nmw...
2026-09-17T16:24:38.665771995Z Wrote inspect data to must-gather.
2026-09-17T16:24:38.761333226Z tar: Removing leading `/' from member names
2026-09-17T16:24:38.778714229Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T16:24:38.784229926Z tar: Removing leading `/' from member names
2026-09-17T16:24:38.809563702Z tar: Removing leading `/' from member names
2026-09-17T16:24:38.810715081Z tar: Removing leading `/' from member names
2026-09-17T16:24:38.827334287Z tar: Removing leading `/' from member names
2026-09-17T16:24:38.831423802Z tar: Removing leading `/' from member names
2026-09-17T16:24:38.865392767Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T16:24:38.954554112Z Wrote inspect data to must-gather.
2026-09-17T16:24:39.084651132Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T16:24:39.225073261Z Wrote inspect data to must-gather.
2026-09-17T16:24:39.355970670Z INFO: Waiting for network log collection to complete ...
2026-09-17T16:24:39.356004898Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T16:24:39.356467518Z INFO: Copying ovnk databases complete.
2026-09-17T16:24:39.358756933Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-09-17T16:24:39.362117965Z ovnk_database_store/
2026-09-17T16:24:39.362168271Z ovnk_database_store/ovnkube-node-z8nmw_sbdb
2026-09-17T16:24:39.390562347Z ovnk_database_store/ovnkube-node-szc22_sbdb
2026-09-17T16:24:39.411691099Z ovnk_database_store/ovnkube-node-z8nmw_nbdb
2026-09-17T16:24:39.412896734Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T16:24:39.418806935Z ovnk_database_store/ovnkube-node-h25hc_sbdb
2026-09-17T16:24:39.442355438Z ovnk_database_store/ovnkube-node-h25hc_nbdb
2026-09-17T16:24:39.450497010Z ovnk_database_store/ovnkube-node-szc22_nbdb
2026-09-17T16:24:39.463167574Z INFO: Network log collection complete.
2026-09-17T16:24:39.488757060Z W0917 16:24:39.488705    1451 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-17T16:24:39.494303575Z Wrote inspect data to must-gather.
2026-09-17T16:24:39.569216646Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T16:24:39.585036931Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:24:39.774050368Z Gathering data for ns/openshift-network-console...
2026-09-17T16:24:39.785250593Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:39.810110411Z INFO: Getting status from alertmanager-main-0
2026-09-17T16:24:39.812598981Z Wrote inspect data to must-gather.
2026-09-17T16:24:40.027246056Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:40.030048624Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T16:24:40.133415512Z Wrote inspect data to must-gather.
2026-09-17T16:24:40.246965228Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:40.279305989Z Gathering data for ns/openshift-console-operator...
2026-09-17T16:24:40.428210909Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T16:24:40.430511300Z INFO: Fallback to identify the container image from release info
2026-09-17T16:24:40.468989648Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:40.472127631Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T16:24:40.665738867Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:40.780491751Z Gathering data for ns/openshift-console...
2026-09-17T16:24:40.868097545Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:40.870775244Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T16:24:40.929513802Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-17T16:24:41.073435242Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T16:24:41.119274946Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:41.213336630Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T16:24:41.368246500Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:41.370884399Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-17T16:24:41.497246313Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T16:24:41.569433155Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:41.795232412Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:41.799122084Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-17T16:24:41.817350568Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-09-17T16:24:41.981395200Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:42.157289544Z Gathering data for ns/openshift-dns-operator...
2026-09-17T16:24:42.196716659Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:42.199186303Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T16:24:42.341195897Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T16:24:42.382618626Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:42.505703179Z Gathering data for ns/openshift-dns...
2026-09-17T16:24:42.581194137Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:24:43.193243606Z Gathering data for ns/openshift-image-registry...
2026-09-17T16:24:43.266456627Z Done executing Istio gather script
2026-09-17T16:24:43.472099909Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T16:24:43.787579463Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T16:24:44.124491354Z Gathering data for ns/openshift-ingress...
2026-09-17T16:24:44.492007895Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T16:24:44.587354322Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T16:24:44.899238489Z Gathering data for ns/openshift-insights...
2026-09-17T16:24:45.707056077Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T16:24:46.822671645Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-09-17T16:24:46.826943438Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T16:24:47.459804555Z Gathering data for ns/kserve...
2026-09-17T16:24:47.950842124Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T16:24:49.069873932Z Waiting for performance profile collector pods to become ready: 8
2026-09-17T16:24:49.210854778Z Gathering data for ns/openshift-config...
2026-09-17T16:24:49.484921690Z Gathering data for ns/openshift-config-managed...
2026-09-17T16:24:49.921221986Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T16:24:50.198816986Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T16:24:50.267144590Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T16:24:50.477531211Z Collecting performance related data for node ip-10-0-128-191.ec2.internal
2026-09-17T16:24:50.488263692Z Collecting performance related data for node ip-10-0-131-144.ec2.internal
2026-09-17T16:24:50.493864983Z Collecting performance related data for node ip-10-0-128-128.ec2.internal
2026-09-17T16:24:50.623325530Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T16:24:51.042212753Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T16:24:51.503898011Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T16:24:51.829964754Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-09-17T16:24:51.905533350Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T16:24:52.386480219Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T16:24:52.796993570Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T16:24:53.182588852Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T16:24:53.812477473Z Gathering data for ns/openshift-multus...
2026-09-17T16:24:55.382132042Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T16:24:56.834902647Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-09-17T16:24:56.878578194Z Gathering data for ns/openshift-host-network...
2026-09-17T16:24:57.235346994Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T16:24:57.849342241Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T16:24:58.329002715Z Gathering data for ns/openshift-network-operator...
2026-09-17T16:24:58.528221198Z Collecting kubelet logs for node ip-10-0-128-191.ec2.internal
2026-09-17T16:24:58.529925300Z Collecting kubelet logs for node ip-10-0-131-144.ec2.internal
2026-09-17T16:24:58.531514754Z Collecting kubelet logs for node ip-10-0-128-128.ec2.internal
2026-09-17T16:24:58.784282318Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T16:24:58.923058778Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vp8j5 namespace
2026-09-17T16:24:58.928006142Z INFO: Node performance data collection complete.
2026-09-17T16:24:59.117083150Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T16:24:59.637673139Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T16:24:59.964896567Z Gathering data for ns/openshift-apiserver...
2026-09-17T16:25:00.430969308Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T16:25:00.749816137Z Gathering data for ns/openshift-controller-manager...
2026-09-17T16:25:01.121214614Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T16:25:01.840017656Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-09-17T16:25:01.873238695Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T16:25:02.192684264Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T16:25:02.533184848Z Gathering data for ns/openshift-service-ca...
2026-09-17T16:25:03.030324267Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T16:25:03.946296358Z Wrote inspect data to must-gather.
2026-09-17T16:25:03.946345624Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T16:25:03.946345624Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T16:25:04.076216426Z error: the server doesn't have a resource type "clusters"
2026-09-17T16:25:04.215457471Z Caches written to disk