2026-07-30T13:56:01.609767830Z [disk usage checker] Started
2026-07-30T13:56:01.613208500Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-30T13:56:01.789468906Z Gathering data for ns/openshift-cluster-version...
2026-07-30T13:56:01.910231200Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T13:56:01.991209562Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T13:56:02.066330996Z Gathering data for ns/default...
2026-07-30T13:56:02.334329875Z Gathering data for ns/openshift...
2026-07-30T13:56:02.712316145Z Gathering data for ns/kube-system...
2026-07-30T13:56:03.354026715Z Gathering data for ns/openshift-etcd...
2026-07-30T13:56:03.626491146Z Wrote inspect data to must-gather.
2026-07-30T13:56:03.626523159Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T13:56:03.626523159Z     namespaces "assisted-installer" not found
2026-07-30T13:56:04.043197704Z Waiting on subprocesses to finish execution.
2026-07-30T13:56:04.050094248Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-30T13:56:04.054430472Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-30T13:56:04.057632295Z INFO: Collecting host service logs for crio
2026-07-30T13:56:04.058148864Z INFO: Collecting host service logs for kubelet
2026-07-30T13:56:04.058519146Z INFO: Collecting host service logs for rpm-ostreed
2026-07-30T13:56:04.058899342Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-30T13:56:04.060109777Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-30T13:56:04.060480174Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-30T13:56:04.060809761Z INFO: Collecting host service logs for NetworkManager
2026-07-30T13:56:04.061136954Z INFO: Collecting host service logs for openvswitch
2026-07-30T13:56:04.061530753Z INFO: Collecting host service logs for ovs-configuration
2026-07-30T13:56:04.061884563Z INFO: Collecting host service logs for ovsdb-server
2026-07-30T13:56:04.062215699Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-30T13:56:04.062629539Z INFO: Waiting for worker host service log collection to complete ...
2026-07-30T13:56:04.070203389Z Executing Istio gather script
2026-07-30T13:56:04.070563921Z INFO: Gathering HAProxy config files
2026-07-30T13:56:04.099025453Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-30T13:56:04.316242069Z No resources found
2026-07-30T13:56:04.330967804Z INFO: Waiting for node performance related collection to complete ...
2026-07-30T13:56:04.772760139Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-30T13:56:04.772849546Z INFO: on-disk MachineConfig config collection complete.
2026-07-30T13:56:04.972604461Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-30T13:56:04.988637369Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-hpq8k
2026-07-30T13:56:04.988637369Z insights-runtime-extractor-j9ljp
2026-07-30T13:56:04.988637369Z insights-runtime-extractor-qqdn7
2026-07-30T13:56:04.988637369Z insights-runtime-extractor-v7wzc
2026-07-30T13:56:05.201810818Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-30T13:56:05.417068043Z INFO: "metallb-operator" not detected. Skipping.
2026-07-30T13:56:05.417338721Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-30T13:56:05.429509548Z Wrote inspect data to must-gather.
2026-07-30T13:56:05.445077276Z No resources found in openshift-etcd namespace.
2026-07-30T13:56:05.546526692Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-30T13:56:05.560028522Z error: the server doesn't have a resource type "performanceprofile"
2026-07-30T13:56:05.619150926Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-30T13:56:05.641964251Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-30T13:56:05.649372026Z INFO: Worker host service log collection to complete.
2026-07-30T13:56:06.012667167Z Wrote inspect data to must-gather.
2026-07-30T13:56:06.169767597Z Wrote inspect data to must-gather.
2026-07-30T13:56:06.193273946Z ERROR: No running kube-apiserver pods found
2026-07-30T13:56:06.264144178Z tar: Removing leading `/' from member names
2026-07-30T13:56:06.274775193Z INFO: HAProxy config collection complete.
2026-07-30T13:56:06.334427546Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-30T13:56:06.407055670Z Wrote inspect data to must-gather.
2026-07-30T13:56:06.459421412Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-30T13:56:06.459421412Z See 'oc rsync -h' for help and examples
2026-07-30T13:56:06.562542250Z No resources found
2026-07-30T13:56:06.567652504Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T13:56:06.636662750Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-30T13:56:06.663804933Z Gathering data for ns/openshift-monitoring...
2026-07-30T13:56:06.756882394Z error: the server doesn't have a resource type "machineconfigs"
2026-07-30T13:56:06.762282639Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-30T13:56:06.785584063Z Wrote inspect data to must-gather.
2026-07-30T13:56:06.865445512Z Wrote inspect data to must-gather.
2026-07-30T13:56:06.865642697Z INFO: Getting rules from prometheus-k8s-0
2026-07-30T13:56:06.907427250Z INFO: 'previous-logs' folder not found on ip-10-0-132-75.ec2.internal, skipping...
2026-07-30T13:56:06.945871847Z Wrote inspect data to must-gather.
2026-07-30T13:56:06.953582265Z Wrote inspect data to must-gather.
2026-07-30T13:56:06.958912020Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-30T13:56:07.189370063Z Wrote inspect data to must-gather.
2026-07-30T13:56:07.191986348Z INFO: 'previous-logs' folder not found on ip-10-0-133-96.ec2.internal, skipping...
2026-07-30T13:56:07.216097023Z Wrote inspect data to must-gather.
2026-07-30T13:56:07.288327118Z INFO: Getting status/config from prometheus-k8s-0
2026-07-30T13:56:07.352781796Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-30T13:56:07.461819890Z INFO: 'previous-logs' folder not found on ip-10-0-141-140.ec2.internal, skipping...
2026-07-30T13:56:07.461858379Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-30T13:56:07.461858379Z INFO: Machine Config Daemon termination log collection complete.
2026-07-30T13:56:07.491336615Z Wrote inspect data to must-gather.
2026-07-30T13:56:07.581490801Z Wrote inspect data to must-gather.
2026-07-30T13:56:07.697493149Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-30T13:56:07.737922311Z INFO: INTERCONNECT MODE
2026-07-30T13:56:07.737959665Z INFO: Gathering ovn-kubernetes DBs
2026-07-30T13:56:07.798683174Z Wrote inspect data to must-gather.
2026-07-30T13:56:07.890007137Z INFO: Gathering OVN_Northbound from ovnkube-node-2trbl...
2026-07-30T13:56:07.890201889Z INFO: Gathering OVN_Northbound from ovnkube-node-mmbms...
2026-07-30T13:56:07.890439770Z INFO: Gathering OVN_Northbound from ovnkube-node-skph2...
2026-07-30T13:56:07.890746511Z INFO: Gathering OVN_Southbound from ovnkube-node-2trbl...
2026-07-30T13:56:07.890975524Z INFO: Gathering OVN_Southbound from ovnkube-node-mmbms...
2026-07-30T13:56:07.891275499Z INFO: Gathering OVN_Southbound from ovnkube-node-skph2...
2026-07-30T13:56:07.953350139Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-30T13:56:08.113509241Z Wrote inspect data to must-gather.
2026-07-30T13:56:08.132380356Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-30T13:56:08.254998748Z tar: Removing leading `/' from member names
2026-07-30T13:56:08.268669306Z tar: Removing leading `/' from member names
2026-07-30T13:56:08.294347076Z tar: Removing leading `/' from member names
2026-07-30T13:56:08.301132048Z tar: Removing leading `/' from member names
2026-07-30T13:56:08.316491118Z tar: Removing leading `/' from member names
2026-07-30T13:56:08.334519303Z tar: Removing leading `/' from member names
2026-07-30T13:56:08.376119777Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:08.386062936Z Wrote inspect data to must-gather.
2026-07-30T13:56:08.411743445Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T13:56:08.425875713Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-30T13:56:08.606789535Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:08.609562354Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-30T13:56:08.618565944Z Wrote inspect data to must-gather.
2026-07-30T13:56:08.707510027Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-30T13:56:08.709175578Z INFO: Waiting for network log collection to complete ...
2026-07-30T13:56:08.709223410Z INFO: Waiting for ovnk database copies to complete ...
2026-07-30T13:56:08.709753291Z INFO: Copying ovnk databases complete.
2026-07-30T13:56:08.711652049Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-30T13:56:08.713967814Z ovnk_database_store/
2026-07-30T13:56:08.713967814Z ovnk_database_store/ovnkube-node-2trbl_sbdb
2026-07-30T13:56:08.731611405Z ovnk_database_store/ovnkube-node-2trbl_nbdb
2026-07-30T13:56:08.738890747Z ovnk_database_store/ovnkube-node-skph2_sbdb
2026-07-30T13:56:08.754413732Z ovnk_database_store/ovnkube-node-skph2_nbdb
2026-07-30T13:56:08.760831355Z ovnk_database_store/ovnkube-node-mmbms_nbdb
2026-07-30T13:56:08.768936208Z ovnk_database_store/ovnkube-node-mmbms_sbdb
2026-07-30T13:56:08.790932868Z INFO: Network log collection complete.
2026-07-30T13:56:08.836459492Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:08.856391409Z Wrote inspect data to must-gather.
2026-07-30T13:56:09.027423847Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:09.031091238Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-30T13:56:09.079987539Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T13:56:09.083919245Z INFO: Getting status from alertmanager-main-0
2026-07-30T13:56:09.152248778Z Wrote inspect data to must-gather.
2026-07-30T13:56:09.231291792Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:09.280681122Z Gathering data for ns/openshift-network-console...
2026-07-30T13:56:09.414355303Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-30T13:56:09.416696955Z INFO: Fallback to identify the container image from release info
2026-07-30T13:56:09.435152359Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:09.437576139Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-30T13:56:09.651320732Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:09.691715768Z Gathering data for ns/openshift-console-operator...
2026-07-30T13:56:09.836313518Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:09.838455870Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-30T13:56:09.975228470Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-30T13:56:10.015129378Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:10.087816432Z Gathering data for ns/openshift-console...
2026-07-30T13:56:10.104719910Z daemonset.apps/perf-node-gather-daemonset created
2026-07-30T13:56:10.227665207Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:56:10.245242661Z Waiting for performance profile collector pods to become ready: 1
2026-07-30T13:56:10.619541657Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-30T13:56:10.916155127Z Done executing Istio gather script
2026-07-30T13:56:11.075552729Z Gathering data for ns/openshift-dns-operator...
2026-07-30T13:56:11.315545333Z Gathering data for ns/openshift-dns...
2026-07-30T13:56:11.356128414Z Waiting for performance profile collector pods to become ready: 2
2026-07-30T13:56:11.660758684Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-30T13:56:11.991572696Z Gathering data for ns/openshift-image-registry...
2026-07-30T13:56:12.462227704Z Gathering data for ns/openshift-ingress-operator...
2026-07-30T13:56:12.474713570Z Waiting for performance profile collector pods to become ready: 3
2026-07-30T13:56:12.741396904Z Gathering data for ns/openshift-ingress...
2026-07-30T13:56:13.037786095Z Gathering data for ns/openshift-ingress-canary...
2026-07-30T13:56:13.375986351Z Gathering data for ns/openshift-insights...
2026-07-30T13:56:13.576921095Z Waiting for performance profile collector pods to become ready: 4
2026-07-30T13:56:14.686239028Z Waiting for performance profile collector pods to become ready: 5
2026-07-30T13:56:15.192199525Z Gathering data for ns/openshift-jobset-operator...
2026-07-30T13:56:15.792564746Z Waiting for performance profile collector pods to become ready: 6
2026-07-30T13:56:15.936834052Z Gathering data for ns/openshift-config...
2026-07-30T13:56:16.215495611Z Gathering data for ns/openshift-config-managed...
2026-07-30T13:56:16.601799347Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-30T13:56:16.665267016Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-30T13:56:16.838915194Z Gathering data for ns/openshift-kube-apiserver...
2026-07-30T13:56:16.915622810Z Waiting for performance profile collector pods to become ready: 7
2026-07-30T13:56:17.074124472Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-30T13:56:17.367364239Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-30T13:56:17.666736206Z Gathering data for ns/openshift-kube-scheduler...
2026-07-30T13:56:17.914595368Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-30T13:56:18.032293006Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-30T13:56:18.247202684Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-30T13:56:18.267764213Z Collecting performance related data for node ip-10-0-141-140.ec2.internal
2026-07-30T13:56:18.274347784Z Collecting performance related data for node ip-10-0-133-96.ec2.internal
2026-07-30T13:56:18.281588463Z Collecting performance related data for node ip-10-0-132-75.ec2.internal
2026-07-30T13:56:18.624819419Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-30T13:56:18.978637161Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-30T13:56:19.650520043Z Gathering data for ns/openshift-multus...
2026-07-30T13:56:21.130382036Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-30T13:56:21.669845257Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-30T13:56:22.289188162Z Gathering data for ns/openshift-host-network...
2026-07-30T13:56:22.577595029Z Gathering data for ns/openshift-network-diagnostics...
2026-07-30T13:56:23.085362325Z Gathering data for ns/openshift-network-node-identity...
2026-07-30T13:56:23.464494660Z Gathering data for ns/openshift-network-operator...
2026-07-30T13:56:23.807201808Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-30T13:56:24.094220128Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-30T13:56:24.543448842Z Gathering data for ns/openshift-apiserver-operator...
2026-07-30T13:56:24.793876457Z Gathering data for ns/openshift-apiserver...
2026-07-30T13:56:25.129928582Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-30T13:56:25.371714971Z Gathering data for ns/openshift-controller-manager...
2026-07-30T13:56:25.618538251Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-30T13:56:26.141443677Z Collecting kubelet logs for node ip-10-0-141-140.ec2.internal
2026-07-30T13:56:26.143218028Z Collecting kubelet logs for node ip-10-0-133-96.ec2.internal
2026-07-30T13:56:26.147958148Z Collecting kubelet logs for node ip-10-0-132-75.ec2.internal
2026-07-30T13:56:26.262149756Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-30T13:56:26.478167615Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-44q57 namespace
2026-07-30T13:56:26.483338610Z INFO: Node performance data collection complete.
2026-07-30T13:56:26.569434614Z Gathering data for ns/openshift-service-ca-operator...
2026-07-30T13:56:26.674366157Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-07-30T13:56:26.865467417Z Gathering data for ns/openshift-service-ca...
2026-07-30T13:56:27.221910042Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-30T13:56:28.035169282Z Wrote inspect data to must-gather.
2026-07-30T13:56:28.035214595Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T13:56:28.035214595Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-30T13:56:28.152675485Z error: the server doesn't have a resource type "clusters"
2026-07-30T13:56:28.276973587Z Caches written to disk