2026-06-09T15:33:42.524251846Z [disk usage checker] Started
2026-06-09T15:33:42.528052328Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T15:33:42.721916909Z Gathering data for ns/openshift-cluster-version...
2026-06-09T15:33:42.900943091Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T15:33:43.004131405Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T15:33:43.085326512Z Gathering data for ns/default...
2026-06-09T15:33:43.400285442Z Gathering data for ns/openshift...
2026-06-09T15:33:43.842837158Z Gathering data for ns/kube-system...
2026-06-09T15:33:44.555963878Z Gathering data for ns/openshift-etcd...
2026-06-09T15:33:44.891270659Z Wrote inspect data to must-gather.
2026-06-09T15:33:44.891305333Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T15:33:44.891305333Z     namespaces "assisted-installer" not found
2026-06-09T15:33:45.065910887Z Waiting on subprocesses to finish execution.
2026-06-09T15:33:45.074258570Z INFO: Collecting host service logs for crio
2026-06-09T15:33:45.074617156Z INFO: Collecting host service logs for kubelet
2026-06-09T15:33:45.074965399Z INFO: Collecting host service logs for rpm-ostreed
2026-06-09T15:33:45.075324467Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-09T15:33:45.075672348Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-09T15:33:45.076011315Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-09T15:33:45.076353616Z INFO: Collecting host service logs for NetworkManager
2026-06-09T15:33:45.076690279Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-09T15:33:45.076690279Z INFO: Collecting host service logs for openvswitch
2026-06-09T15:33:45.076838321Z Executing Istio gather script
2026-06-09T15:33:45.077011347Z INFO: Collecting host service logs for ovs-configuration
2026-06-09T15:33:45.077471265Z INFO: Collecting host service logs for ovsdb-server
2026-06-09T15:33:45.077808784Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-09T15:33:45.078207100Z INFO: Waiting for worker host service log collection to complete ...
2026-06-09T15:33:45.085594748Z INFO: Gathering HAProxy config files
2026-06-09T15:33:45.092966253Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-09T15:33:45.093939359Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-09T15:33:45.100678634Z INFO: Waiting for node performance related collection to complete ...
2026-06-09T15:33:45.669657563Z No resources found in openshift-etcd namespace.
2026-06-09T15:33:45.777440223Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-09T15:33:45.777471065Z INFO: on-disk MachineConfig config collection complete.
2026-06-09T15:33:45.803646787Z INFO: "metallb-operator" not detected. Skipping.
2026-06-09T15:33:46.000824130Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-09T15:33:46.204864133Z error: the server doesn't have a resource type "performanceprofile"
2026-06-09T15:33:46.271130124Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-09T15:33:46.406193074Z ERROR: No running kube-apiserver pods found
2026-06-09T15:33:46.528419128Z INFO: Collecting Insights Archives from insights-operator-896549976-6x9fn
2026-06-09T15:33:46.528481522Z insights-runtime-extractor-d7499
2026-06-09T15:33:46.528514208Z insights-runtime-extractor-h4rg9
2026-06-09T15:33:46.528534769Z insights-runtime-extractor-krddf
2026-06-09T15:33:46.610899352Z No resources found
2026-06-09T15:33:46.718866885Z Wrote inspect data to must-gather.
2026-06-09T15:33:46.732358851Z INFO: Worker host service log collection to complete.
2026-06-09T15:33:46.754152724Z Wrote inspect data to must-gather.
2026-06-09T15:33:46.778477271Z Wrote inspect data to must-gather.
2026-06-09T15:33:46.804137632Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-09T15:33:46.867233891Z Gathering data for ns/openshift-monitoring...
2026-06-09T15:33:47.051293471Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-09T15:33:47.051293471Z See 'oc rsync -h' for help and examples
2026-06-09T15:33:47.094361517Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-09T15:33:47.103085330Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T15:33:47.133421185Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-09T15:33:47.188174209Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-09T15:33:47.370836341Z No resources found
2026-06-09T15:33:47.470058352Z Wrote inspect data to must-gather.
2026-06-09T15:33:47.541391597Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-09T15:33:47.563770689Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T15:33:47.596246889Z Wrote inspect data to must-gather.
2026-06-09T15:33:47.611739032Z error: the server doesn't have a resource type "machineconfigs"
2026-06-09T15:33:47.698073613Z Wrote inspect data to must-gather.
2026-06-09T15:33:47.709465766Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-09T15:33:47.720006837Z Wrote inspect data to must-gather.
2026-06-09T15:33:47.734170077Z INFO: 'previous-logs' folder not found on ip-10-0-131-77.ec2.internal, skipping...
2026-06-09T15:33:47.849397065Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-09T15:33:47.896292146Z tar: Removing leading `/' from member names
2026-06-09T15:33:47.899692805Z INFO: HAProxy config collection complete.
2026-06-09T15:33:47.998862285Z Wrote inspect data to must-gather.
2026-06-09T15:33:47.999361817Z INFO: 'previous-logs' folder not found on ip-10-0-139-10.ec2.internal, skipping...
2026-06-09T15:33:48.066153479Z INFO: Getting rules from prometheus-k8s-0
2026-06-09T15:33:48.091313510Z Wrote inspect data to must-gather.
2026-06-09T15:33:48.260441710Z Wrote inspect data to must-gather.
2026-06-09T15:33:48.270912821Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-09T15:33:48.271949347Z INFO: 'previous-logs' folder not found on ip-10-0-140-2.ec2.internal, skipping...
2026-06-09T15:33:48.272048707Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-09T15:33:48.272101105Z INFO: Machine Config Daemon termination log collection complete.
2026-06-09T15:33:48.487191726Z Wrote inspect data to must-gather.
2026-06-09T15:33:48.503331217Z INFO: Getting status/config from prometheus-k8s-0
2026-06-09T15:33:48.513405041Z Wrote inspect data to must-gather.
2026-06-09T15:33:48.681997334Z Wrote inspect data to must-gather.
2026-06-09T15:33:48.798345077Z INFO: INTERCONNECT MODE
2026-06-09T15:33:48.798371770Z INFO: Gathering ovn-kubernetes DBs
2026-06-09T15:33:48.914588256Z Wrote inspect data to must-gather.
2026-06-09T15:33:48.966094111Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-09T15:33:49.001376959Z INFO: Gathering OVN_Northbound from ovnkube-node-b8lgv...
2026-06-09T15:33:49.001949941Z INFO: Gathering OVN_Northbound from ovnkube-node-jdfzs...
2026-06-09T15:33:49.004140014Z INFO: Gathering OVN_Northbound from ovnkube-node-qkrvt...
2026-06-09T15:33:49.004460294Z INFO: Gathering OVN_Southbound from ovnkube-node-b8lgv...
2026-06-09T15:33:49.004708309Z INFO: Gathering OVN_Southbound from ovnkube-node-jdfzs...
2026-06-09T15:33:49.004957918Z INFO: Gathering OVN_Southbound from ovnkube-node-qkrvt...
2026-06-09T15:33:49.023180444Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-09T15:33:49.050960081Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T15:33:49.213087501Z Wrote inspect data to must-gather.
2026-06-09T15:33:49.322875217Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-09T15:33:49.363692350Z tar: Removing leading `/' from member names
2026-06-09T15:33:49.384933925Z tar: Removing leading `/' from member names
2026-06-09T15:33:49.398878178Z tar: Removing leading `/' from member names
2026-06-09T15:33:49.401038683Z tar: Removing leading `/' from member names
2026-06-09T15:33:49.416639195Z tar: Removing leading `/' from member names
2026-06-09T15:33:49.445536848Z tar: Removing leading `/' from member names
2026-06-09T15:33:49.450696985Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:49.517095187Z Wrote inspect data to must-gather.
2026-06-09T15:33:49.633951172Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-09T15:33:49.702398270Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:49.705303229Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-09T15:33:49.781954615Z W0609 15:33:49.781900    1424 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-09T15:33:49.787294519Z Wrote inspect data to must-gather.
2026-06-09T15:33:49.878324656Z INFO: Waiting for network log collection to complete ...
2026-06-09T15:33:49.878439828Z INFO: Waiting for ovnk database copies to complete ...
2026-06-09T15:33:49.878954956Z INFO: Copying ovnk databases complete.
2026-06-09T15:33:49.880940201Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-06-09T15:33:49.883908999Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T15:33:49.884392113Z ovnk_database_store/
2026-06-09T15:33:49.884392113Z ovnk_database_store/ovnkube-node-jdfzs_nbdb
2026-06-09T15:33:49.892233805Z ovnk_database_store/ovnkube-node-b8lgv_nbdb
2026-06-09T15:33:49.903117283Z ovnk_database_store/ovnkube-node-qkrvt_sbdb
2026-06-09T15:33:49.924645583Z ovnk_database_store/ovnkube-node-b8lgv_sbdb
2026-06-09T15:33:49.931564670Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-09T15:33:49.943506702Z ovnk_database_store/ovnkube-node-qkrvt_nbdb
2026-06-09T15:33:49.952621033Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:49.952729592Z ovnk_database_store/ovnkube-node-jdfzs_sbdb
2026-06-09T15:33:49.973808614Z INFO: Network log collection complete.
2026-06-09T15:33:50.115408067Z Gathering data for ns/openshift-network-console...
2026-06-09T15:33:50.121406370Z Wrote inspect data to must-gather.
2026-06-09T15:33:50.168623948Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:50.173285178Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-09T15:33:50.345691646Z INFO: Getting status from alertmanager-main-0
2026-06-09T15:33:50.381396856Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:50.401268389Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-09T15:33:50.403570345Z INFO: Fallback to identify the container image from release info
2026-06-09T15:33:50.594346103Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:50.596488941Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-09T15:33:50.631978141Z Gathering data for ns/openshift-console-operator...
2026-06-09T15:33:50.798892234Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:50.978336992Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:50.980579816Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-09T15:33:51.120668276Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-09T15:33:51.124049449Z Gathering data for ns/openshift-console...
2026-06-09T15:33:51.147287435Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:51.245806955Z daemonset.apps/perf-node-gather-daemonset created
2026-06-09T15:33:51.350119671Z Wrote inspect data to /must-gather/istio.
2026-06-09T15:33:51.371811728Z Waiting for performance profile collector pods to become ready: 1
2026-06-09T15:33:51.725495120Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-09T15:33:52.021259414Z Done executing Istio gather script
2026-06-09T15:33:52.259958145Z Gathering data for ns/openshift-dns-operator...
2026-06-09T15:33:52.490916366Z Waiting for performance profile collector pods to become ready: 2
2026-06-09T15:33:52.555499427Z Gathering data for ns/openshift-dns...
2026-06-09T15:33:52.582590014Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T15:33:53.183366277Z Gathering data for ns/openshift-image-registry...
2026-06-09T15:33:53.604258026Z Waiting for performance profile collector pods to become ready: 3
2026-06-09T15:33:53.684961746Z Gathering data for ns/openshift-ingress-operator...
2026-06-09T15:33:54.031742022Z Gathering data for ns/openshift-ingress...
2026-06-09T15:33:54.371161961Z Gathering data for ns/openshift-ingress-canary...
2026-06-09T15:33:54.705893904Z Waiting for performance profile collector pods to become ready: 4
2026-06-09T15:33:54.863151424Z Gathering data for ns/openshift-insights...
2026-06-09T15:33:55.814864460Z Waiting for performance profile collector pods to become ready: 5
2026-06-09T15:33:56.827205797Z Gathering data for ns/openshift-jobset-operator...
2026-06-09T15:33:56.926180242Z Waiting for performance profile collector pods to become ready: 6
2026-06-09T15:33:57.587050976Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T15:33:57.702062554Z Gathering data for ns/openshift-config...
2026-06-09T15:33:57.989010184Z Gathering data for ns/openshift-config-managed...
2026-06-09T15:33:58.056615549Z Waiting for performance profile collector pods to become ready: 7
2026-06-09T15:33:58.467155406Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-09T15:33:58.767902819Z Gathering data for ns/openshift-kube-apiserver...
2026-06-09T15:33:59.070160367Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-09T15:33:59.177160965Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-09T15:33:59.421744937Z Collecting performance related data for node ip-10-0-139-10.ec2.internal
2026-06-09T15:33:59.429881744Z Collecting performance related data for node ip-10-0-140-2.ec2.internal
2026-06-09T15:33:59.437687982Z Collecting performance related data for node ip-10-0-131-77.ec2.internal
2026-06-09T15:33:59.460463450Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-09T15:33:59.840931304Z Gathering data for ns/openshift-kube-scheduler...
2026-06-09T15:34:00.215877565Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-09T15:34:00.577904983Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-09T15:34:01.003849889Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-09T15:34:01.390874408Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-09T15:34:01.986580219Z Gathering data for ns/openshift-multus...
2026-06-09T15:34:02.591815856Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T15:34:03.869007932Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-09T15:34:05.406278275Z Gathering data for ns/openshift-host-network...
2026-06-09T15:34:05.763373924Z Gathering data for ns/openshift-network-diagnostics...
2026-06-09T15:34:06.419094461Z Gathering data for ns/openshift-network-node-identity...
2026-06-09T15:34:06.919979155Z Gathering data for ns/openshift-network-operator...
2026-06-09T15:34:07.240801153Z Collecting kubelet logs for node ip-10-0-139-10.ec2.internal
2026-06-09T15:34:07.242608374Z Collecting kubelet logs for node ip-10-0-140-2.ec2.internal
2026-06-09T15:34:07.244757106Z Collecting kubelet logs for node ip-10-0-131-77.ec2.internal
2026-06-09T15:34:07.415269152Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-09T15:34:07.595291111Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-7rbrl namespace
2026-06-09T15:34:07.596291898Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-09T15:34:07.600302208Z INFO: Node performance data collection complete.
2026-06-09T15:34:07.751906307Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-09T15:34:08.265911066Z Gathering data for ns/openshift-apiserver-operator...
2026-06-09T15:34:08.584408375Z Gathering data for ns/openshift-apiserver...
2026-06-09T15:34:09.005135735Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-09T15:34:09.283001688Z Gathering data for ns/openshift-controller-manager...
2026-06-09T15:34:09.574491913Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-09T15:34:10.263914484Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-09T15:34:10.626719886Z Gathering data for ns/openshift-service-ca-operator...
2026-06-09T15:34:10.973123938Z Gathering data for ns/openshift-service-ca...
2026-06-09T15:34:11.381952885Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-09T15:34:12.316654199Z Wrote inspect data to must-gather.
2026-06-09T15:34:12.316697730Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T15:34:12.316697730Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-09T15:34:12.437945861Z error: the server doesn't have a resource type "clusters"
2026-06-09T15:34:12.572958047Z Caches written to disk