2026-09-07T14:19:27.390932676Z [disk usage checker] Started
2026-09-07T14:19:27.394405697Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:19:27.591058250Z Gathering data for ns/openshift-cluster-version...
2026-09-07T14:19:27.721537941Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T14:19:27.818019019Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T14:19:27.910047644Z Gathering data for ns/default...
2026-09-07T14:19:28.222658145Z Gathering data for ns/openshift...
2026-09-07T14:19:28.644200581Z Gathering data for ns/kube-system...
2026-09-07T14:19:29.348099721Z Gathering data for ns/openshift-etcd...
2026-09-07T14:19:29.746405488Z Wrote inspect data to must-gather.
2026-09-07T14:19:29.746547615Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T14:19:29.746547615Z     namespaces "assisted-installer" not found
2026-09-07T14:19:30.060786421Z Waiting on subprocesses to finish execution.
2026-09-07T14:19:30.071350252Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-07T14:19:30.079608781Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-07T14:19:30.079839812Z INFO: Gathering HAProxy config files
2026-09-07T14:19:30.086560382Z Executing Istio gather script
2026-09-07T14:19:30.120434447Z INFO: Collecting host service logs for crio
2026-09-07T14:19:30.120840656Z INFO: Collecting host service logs for kubelet
2026-09-07T14:19:30.121239358Z INFO: Collecting host service logs for rpm-ostreed
2026-09-07T14:19:30.121431853Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-07T14:19:30.121697289Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-07T14:19:30.122095621Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-07T14:19:30.122502536Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-07T14:19:30.122876422Z INFO: Collecting host service logs for NetworkManager
2026-09-07T14:19:30.123289191Z INFO: Collecting host service logs for openvswitch
2026-09-07T14:19:30.123673428Z INFO: Collecting host service logs for ovs-configuration
2026-09-07T14:19:30.124037567Z INFO: Collecting host service logs for ovsdb-server
2026-09-07T14:19:30.124422508Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-07T14:19:30.124864995Z INFO: Waiting for worker host service log collection to complete ...
2026-09-07T14:19:30.221525592Z INFO: Waiting for node performance related collection to complete ...
2026-09-07T14:19:30.504273207Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-07T14:19:30.562226326Z No resources found in openshift-etcd namespace.
2026-09-07T14:19:30.823380179Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-07T14:19:30.837448947Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-07T14:19:31.258963525Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-07T14:19:31.259656494Z No resources found
2026-09-07T14:19:31.285468550Z error: the server doesn't have a resource type "performanceprofile"
2026-09-07T14:19:31.353142950Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-tb428
2026-09-07T14:19:31.353207002Z insights-runtime-extractor-7m4sx
2026-09-07T14:19:31.353228432Z insights-runtime-extractor-h9l2d
2026-09-07T14:19:31.353247733Z insights-runtime-extractor-svglx
2026-09-07T14:19:31.363953702Z INFO: "metallb-operator" not detected. Skipping.
2026-09-07T14:19:31.475585728Z INFO: Worker host service log collection to complete.
2026-09-07T14:19:31.529324847Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-07T14:19:31.529431741Z INFO: on-disk MachineConfig config collection complete.
2026-09-07T14:19:31.684892468Z Wrote inspect data to must-gather.
2026-09-07T14:19:31.759343836Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-07T14:19:31.865823777Z Wrote inspect data to must-gather.
2026-09-07T14:19:31.994145322Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-07T14:19:32.029675730Z No resources found
2026-09-07T14:19:32.123244328Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-07T14:19:32.123244328Z See 'oc rsync -h' for help and examples
2026-09-07T14:19:32.128051341Z ERROR: No running kube-apiserver pods found
2026-09-07T14:19:32.192609390Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T14:19:32.228420636Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-07T14:19:32.332763357Z Wrote inspect data to must-gather.
2026-09-07T14:19:32.421473606Z Wrote inspect data to must-gather.
2026-09-07T14:19:32.515937908Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:19:32.547553888Z Wrote inspect data to must-gather.
2026-09-07T14:19:32.582608202Z Gathering data for ns/openshift-monitoring...
2026-09-07T14:19:32.652809245Z Wrote inspect data to must-gather.
2026-09-07T14:19:32.704440628Z INFO: 'previous-logs' folder not found on ip-10-0-133-97.ec2.internal, skipping...
2026-09-07T14:19:32.728950444Z Wrote inspect data to must-gather.
2026-09-07T14:19:32.731934079Z tar: Removing leading `/' from member names
2026-09-07T14:19:32.741184369Z INFO: HAProxy config collection complete.
2026-09-07T14:19:32.775701425Z error: the server doesn't have a resource type "machineconfigs"
2026-09-07T14:19:32.940999849Z INFO: Getting rules from prometheus-k8s-0
2026-09-07T14:19:32.988490735Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-07T14:19:32.991957185Z Wrote inspect data to must-gather.
2026-09-07T14:19:33.044890911Z INFO: 'previous-logs' folder not found on ip-10-0-134-113.ec2.internal, skipping...
2026-09-07T14:19:33.259642016Z Wrote inspect data to must-gather.
2026-09-07T14:19:33.305049561Z Wrote inspect data to must-gather.
2026-09-07T14:19:33.359415893Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-07T14:19:33.365913350Z INFO: 'previous-logs' folder not found on ip-10-0-136-203.ec2.internal, skipping...
2026-09-07T14:19:33.365947473Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-07T14:19:33.365981917Z INFO: Machine Config Daemon termination log collection complete.
2026-09-07T14:19:33.393344012Z INFO: Getting status/config from prometheus-k8s-0
2026-09-07T14:19:33.442898164Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-07T14:19:33.578015237Z Wrote inspect data to must-gather.
2026-09-07T14:19:33.652098022Z Wrote inspect data to must-gather.
2026-09-07T14:19:33.808006448Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-07T14:19:33.878678583Z Wrote inspect data to must-gather.
2026-09-07T14:19:33.889211447Z INFO: INTERCONNECT MODE
2026-09-07T14:19:33.889211447Z INFO: Gathering ovn-kubernetes DBs
2026-09-07T14:19:34.036235310Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-07T14:19:34.055061997Z INFO: Gathering OVN_Northbound from ovnkube-node-5vmb8...
2026-09-07T14:19:34.055441042Z INFO: Gathering OVN_Northbound from ovnkube-node-sp2pd...
2026-09-07T14:19:34.055699698Z INFO: Gathering OVN_Northbound from ovnkube-node-tzk66...
2026-09-07T14:19:34.056014499Z INFO: Gathering OVN_Southbound from ovnkube-node-5vmb8...
2026-09-07T14:19:34.056311496Z INFO: Gathering OVN_Southbound from ovnkube-node-sp2pd...
2026-09-07T14:19:34.056584801Z INFO: Gathering OVN_Southbound from ovnkube-node-tzk66...
2026-09-07T14:19:34.164965653Z Wrote inspect data to must-gather.
2026-09-07T14:19:34.370242834Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-07T14:19:34.385952580Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-07T14:19:34.419665655Z tar: Removing leading `/' from member names
2026-09-07T14:19:34.424350176Z tar: Removing leading `/' from member names
2026-09-07T14:19:34.432647123Z tar: Removing leading `/' from member names
2026-09-07T14:19:34.452443413Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-07T14:19:34.460966482Z tar: Removing leading `/' from member names
2026-09-07T14:19:34.462657029Z tar: Removing leading `/' from member names
2026-09-07T14:19:34.472572681Z tar: Removing leading `/' from member names
2026-09-07T14:19:34.561402417Z Wrote inspect data to must-gather.
2026-09-07T14:19:34.626524316Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:34.750485141Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-07T14:19:34.833586705Z Wrote inspect data to must-gather.
2026-09-07T14:19:34.901752416Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:34.906548547Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-07T14:19:34.994360272Z INFO: Waiting for network log collection to complete ...
2026-09-07T14:19:34.994403901Z INFO: Waiting for ovnk database copies to complete ...
2026-09-07T14:19:34.994883486Z INFO: Copying ovnk databases complete.
2026-09-07T14:19:34.996825820Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-09-07T14:19:34.999891561Z ovnk_database_store/
2026-09-07T14:19:34.999946809Z ovnk_database_store/ovnkube-node-sp2pd_sbdb
2026-09-07T14:19:35.020208064Z ovnk_database_store/ovnkube-node-tzk66_nbdb
2026-09-07T14:19:35.029432841Z ovnk_database_store/ovnkube-node-sp2pd_nbdb
2026-09-07T14:19:35.034643213Z ovnk_database_store/ovnkube-node-tzk66_sbdb
2026-09-07T14:19:35.050453757Z ovnk_database_store/ovnkube-node-5vmb8_nbdb
2026-09-07T14:19:35.058037283Z ovnk_database_store/ovnkube-node-5vmb8_sbdb
2026-09-07T14:19:35.070183222Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-07T14:19:35.084786464Z INFO: Network log collection complete.
2026-09-07T14:19:35.126042296Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:35.168201541Z Wrote inspect data to must-gather.
2026-09-07T14:19:35.198046797Z INFO: Getting status from alertmanager-main-0
2026-09-07T14:19:35.311023926Z Gathering data for ns/openshift-network-console...
2026-09-07T14:19:35.357690788Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:35.360263010Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-07T14:19:35.470925010Z Wrote inspect data to must-gather.
2026-09-07T14:19:35.562038273Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:35.731772748Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-07T14:19:35.734592967Z INFO: Fallback to identify the container image from release info
2026-09-07T14:19:35.772907202Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:35.776007214Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-07T14:19:35.932570313Z Gathering data for ns/openshift-console-operator...
2026-09-07T14:19:36.007877332Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:36.199722924Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:36.203228939Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-07T14:19:36.289236336Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-07T14:19:36.386011575Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:36.439821024Z daemonset.apps/perf-node-gather-daemonset created
2026-09-07T14:19:36.503079037Z Gathering data for ns/openshift-console...
2026-09-07T14:19:36.581730805Z Waiting for performance profile collector pods to become ready: 1
2026-09-07T14:19:36.607957710Z Wrote inspect data to /must-gather/istio.
2026-09-07T14:19:37.158621821Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-07T14:19:37.332554667Z Done executing Istio gather script
2026-09-07T14:19:37.523980035Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:19:37.707075254Z Waiting for performance profile collector pods to become ready: 2
2026-09-07T14:19:37.838360722Z Gathering data for ns/openshift-dns-operator...
2026-09-07T14:19:38.186979251Z Gathering data for ns/openshift-dns...
2026-09-07T14:19:38.834476062Z Waiting for performance profile collector pods to become ready: 3
2026-09-07T14:19:38.904872276Z Gathering data for ns/openshift-image-registry...
2026-09-07T14:19:39.424600146Z Gathering data for ns/openshift-ingress-operator...
2026-09-07T14:19:39.861895081Z Gathering data for ns/openshift-ingress...
2026-09-07T14:19:39.963639694Z Waiting for performance profile collector pods to become ready: 4
2026-09-07T14:19:40.304717889Z Gathering data for ns/openshift-ingress-canary...
2026-09-07T14:19:40.901347593Z Gathering data for ns/openshift-insights...
2026-09-07T14:19:41.076705837Z Waiting for performance profile collector pods to become ready: 5
2026-09-07T14:19:42.198380615Z Waiting for performance profile collector pods to become ready: 6
2026-09-07T14:19:42.528593246Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:19:43.325562797Z Waiting for performance profile collector pods to become ready: 7
2026-09-07T14:19:43.619857317Z Gathering data for ns/openshift-jobset-operator...
2026-09-07T14:19:44.440797820Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-07T14:19:44.640001571Z Gathering data for ns/openshift-config...
2026-09-07T14:19:44.704553148Z Collecting performance related data for node ip-10-0-133-97.ec2.internal
2026-09-07T14:19:44.714525464Z Collecting performance related data for node ip-10-0-134-113.ec2.internal
2026-09-07T14:19:44.725638307Z Collecting performance related data for node ip-10-0-136-203.ec2.internal
2026-09-07T14:19:45.196806917Z Gathering data for ns/openshift-config-managed...
2026-09-07T14:19:46.051983164Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-07T14:19:46.616809740Z Gathering data for ns/openshift-kube-apiserver...
2026-09-07T14:19:47.216523837Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-07T14:19:47.533331830Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:19:47.740263686Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-07T14:19:48.389523152Z Gathering data for ns/openshift-kube-scheduler...
2026-09-07T14:19:48.940338806Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-07T14:19:49.460979164Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-07T14:19:50.062162270Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-07T14:19:50.611575179Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-07T14:19:51.439627669Z Gathering data for ns/openshift-multus...
2026-09-07T14:19:52.538272652Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:19:54.048036248Z Collecting kubelet logs for node ip-10-0-133-97.ec2.internal
2026-09-07T14:19:54.051028406Z Collecting kubelet logs for node ip-10-0-134-113.ec2.internal
2026-09-07T14:19:54.053258566Z Collecting kubelet logs for node ip-10-0-136-203.ec2.internal
2026-09-07T14:19:54.216366243Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-07T14:19:54.411607540Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ghs7b namespace
2026-09-07T14:19:54.419914083Z INFO: Node performance data collection complete.
2026-09-07T14:19:56.240983577Z Gathering data for ns/openshift-host-network...
2026-09-07T14:19:56.672434196Z Gathering data for ns/openshift-network-diagnostics...
2026-09-07T14:19:57.315901695Z Gathering data for ns/openshift-network-node-identity...
2026-09-07T14:19:57.543593318Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:19:57.798475529Z Gathering data for ns/openshift-network-operator...
2026-09-07T14:19:58.284101485Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-07T14:19:58.590301919Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-07T14:19:59.219058977Z Gathering data for ns/openshift-apiserver-operator...
2026-09-07T14:19:59.554776525Z Gathering data for ns/openshift-apiserver...
2026-09-07T14:20:00.184469673Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-07T14:20:00.566022751Z Gathering data for ns/openshift-controller-manager...
2026-09-07T14:20:00.949785190Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-07T14:20:01.752334176Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-07T14:20:02.181602404Z Gathering data for ns/openshift-service-ca-operator...
2026-09-07T14:20:02.549606142Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-07T14:20:02.632354508Z Gathering data for ns/openshift-service-ca...
2026-09-07T14:20:03.195941425Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-07T14:20:04.574286325Z Wrote inspect data to must-gather.
2026-09-07T14:20:04.574340685Z error: inspection completed with the errors occurred while gathering data:
2026-09-07T14:20:04.574340685Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-07T14:20:04.705204950Z error: the server doesn't have a resource type "clusters"
2026-09-07T14:20:04.838624383Z Caches written to disk