2026-09-16T19:14:23.944677768Z [disk usage checker] Started
2026-09-16T19:14:23.948082223Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-16T19:14:24.145924295Z Gathering data for ns/openshift-cluster-version...
2026-09-16T19:14:24.293127957Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:14:24.385158619Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:14:24.470272690Z Gathering data for ns/default...
2026-09-16T19:14:24.807050332Z Gathering data for ns/openshift...
2026-09-16T19:14:25.266072653Z Gathering data for ns/kube-system...
2026-09-16T19:14:25.927284278Z Gathering data for ns/openshift-etcd...
2026-09-16T19:14:26.230445938Z Wrote inspect data to must-gather.
2026-09-16T19:14:26.230485403Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:14:26.230485403Z     namespaces "assisted-installer" not found
2026-09-16T19:14:26.703257927Z Waiting on subprocesses to finish execution.
2026-09-16T19:14:26.708192335Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-16T19:14:26.713758488Z INFO: Collecting host service logs for crio
2026-09-16T19:14:26.714032484Z INFO: Collecting host service logs for kubelet
2026-09-16T19:14:26.714453684Z INFO: Collecting host service logs for rpm-ostreed
2026-09-16T19:14:26.714749125Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-16T19:14:26.715075570Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-16T19:14:26.715432399Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-16T19:14:26.715736336Z INFO: Collecting host service logs for NetworkManager
2026-09-16T19:14:26.716056494Z INFO: Collecting host service logs for openvswitch
2026-09-16T19:14:26.716381351Z INFO: Collecting host service logs for ovs-configuration
2026-09-16T19:14:26.716725407Z INFO: Collecting host service logs for ovsdb-server
2026-09-16T19:14:26.717061298Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-16T19:14:26.717456318Z INFO: Waiting for worker host service log collection to complete ...
2026-09-16T19:14:26.720056115Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-16T19:14:26.725333934Z INFO: Gathering HAProxy config files
2026-09-16T19:14:26.732280010Z Executing Istio gather script
2026-09-16T19:14:26.755812587Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-16T19:14:26.762781631Z INFO: Waiting for node performance related collection to complete ...
2026-09-16T19:14:27.096501180Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-16T19:14:27.096603475Z INFO: on-disk MachineConfig config collection complete.
2026-09-16T19:14:27.332254421Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-16T19:14:27.510806017Z No resources found in openshift-etcd namespace.
2026-09-16T19:14:27.918210563Z ERROR: No running kube-apiserver pods found
2026-09-16T19:14:28.033340018Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-16T19:14:28.307994050Z No resources found
2026-09-16T19:14:28.321334887Z error: the server doesn't have a resource type "performanceprofile"
2026-09-16T19:14:28.400280207Z Wrote inspect data to must-gather.
2026-09-16T19:14:28.470141936Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-16T19:14:28.471422766Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-16T19:14:28.513642575Z INFO: Collecting Insights Archives from insights-operator-5844b856db-2jhcw
2026-09-16T19:14:28.513642575Z insights-runtime-extractor-9lbj8
2026-09-16T19:14:28.513642575Z insights-runtime-extractor-l97v6
2026-09-16T19:14:28.513642575Z insights-runtime-extractor-wkl5v
2026-09-16T19:14:28.539550112Z INFO: Worker host service log collection to complete.
2026-09-16T19:14:28.617840429Z Wrote inspect data to must-gather.
2026-09-16T19:14:28.677529526Z Gathering data for ns/openshift-monitoring...
2026-09-16T19:14:28.745787046Z INFO: "metallb-operator" not detected. Skipping.
2026-09-16T19:14:29.034012353Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-16T19:14:29.075682037Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-16T19:14:29.143950396Z Wrote inspect data to must-gather.
2026-09-16T19:14:29.153603230Z No resources found
2026-09-16T19:14:29.154610133Z INFO: 'previous-logs' folder not found on ip-10-0-128-222.ec2.internal, skipping...
2026-09-16T19:14:29.214035824Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:14:29.228467423Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-16T19:14:29.228467423Z See 'oc rsync -h' for help and examples
2026-09-16T19:14:29.251342011Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-16T19:14:29.336482164Z Wrote inspect data to must-gather.
2026-09-16T19:14:29.419372049Z Wrote inspect data to must-gather.
2026-09-16T19:14:29.520409026Z Wrote inspect data to must-gather.
2026-09-16T19:14:29.549816259Z Wrote inspect data to must-gather.
2026-09-16T19:14:29.570612612Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-16T19:14:29.655387040Z tar: Removing leading `/' from member names
2026-09-16T19:14:29.660763548Z INFO: HAProxy config collection complete.
2026-09-16T19:14:29.741976816Z error: the server doesn't have a resource type "machineconfigs"
2026-09-16T19:14:29.747157090Z INFO: 'previous-logs' folder not found on ip-10-0-140-230.ec2.internal, skipping...
2026-09-16T19:14:29.817692875Z Wrote inspect data to must-gather.
2026-09-16T19:14:29.928435826Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-16T19:14:30.028303449Z INFO: 'previous-logs' folder not found on ip-10-0-143-139.ec2.internal, skipping...
2026-09-16T19:14:30.028407896Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-16T19:14:30.028458580Z INFO: Machine Config Daemon termination log collection complete.
2026-09-16T19:14:30.097715679Z INFO: Getting rules from prometheus-k8s-0
2026-09-16T19:14:30.126152615Z Wrote inspect data to must-gather.
2026-09-16T19:14:30.197778670Z Wrote inspect data to must-gather.
2026-09-16T19:14:30.367873239Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-16T19:14:30.402803803Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-16T19:14:30.431250692Z Wrote inspect data to must-gather.
2026-09-16T19:14:30.597270611Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-16T19:14:30.607913367Z INFO: Getting status/config from prometheus-k8s-0
2026-09-16T19:14:30.652696944Z Wrote inspect data to must-gather.
2026-09-16T19:14:30.770156351Z INFO: INTERCONNECT MODE
2026-09-16T19:14:30.770156351Z INFO: Gathering ovn-kubernetes DBs
2026-09-16T19:14:30.909408673Z Wrote inspect data to must-gather.
2026-09-16T19:14:30.953580589Z INFO: Gathering OVN_Northbound from ovnkube-node-69rtg...
2026-09-16T19:14:30.953806987Z INFO: Gathering OVN_Northbound from ovnkube-node-v2qjb...
2026-09-16T19:14:30.954037008Z INFO: Gathering OVN_Northbound from ovnkube-node-wc29d...
2026-09-16T19:14:30.954328356Z INFO: Gathering OVN_Southbound from ovnkube-node-69rtg...
2026-09-16T19:14:30.954620498Z INFO: Gathering OVN_Southbound from ovnkube-node-v2qjb...
2026-09-16T19:14:30.954903826Z INFO: Gathering OVN_Southbound from ovnkube-node-wc29d...
2026-09-16T19:14:31.233123789Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-16T19:14:31.274501822Z Wrote inspect data to must-gather.
2026-09-16T19:14:31.329476269Z tar: Removing leading `/' from member names
2026-09-16T19:14:31.332616865Z tar: Removing leading `/' from member names
2026-09-16T19:14:31.459360155Z tar: Removing leading `/' from member names
2026-09-16T19:14:31.462563426Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-16T19:14:31.499217501Z tar: Removing leading `/' from member names
2026-09-16T19:14:31.527375162Z tar: Removing leading `/' from member names
2026-09-16T19:14:31.598789515Z tar: Removing leading `/' from member names
2026-09-16T19:14:31.623495826Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-16T19:14:31.713984995Z Wrote inspect data to must-gather.
2026-09-16T19:14:31.736777189Z Gathering data for ns/openshift-network-console...
2026-09-16T19:14:31.739635306Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-16T19:14:31.924030302Z INFO: Waiting for network log collection to complete ...
2026-09-16T19:14:31.924088726Z INFO: Waiting for ovnk database copies to complete ...
2026-09-16T19:14:31.924754857Z INFO: Copying ovnk databases complete.
2026-09-16T19:14:31.924786801Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-16T19:14:31.926313402Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-09-16T19:14:31.928450812Z ovnk_database_store/
2026-09-16T19:14:31.928450812Z ovnk_database_store/ovnkube-node-wc29d_sbdb
2026-09-16T19:14:31.950863548Z ovnk_database_store/ovnkube-node-69rtg_sbdb
2026-09-16T19:14:31.965993341Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:31.973348896Z ovnk_database_store/ovnkube-node-wc29d_nbdb
2026-09-16T19:14:31.984923522Z ovnk_database_store/ovnkube-node-v2qjb_sbdb
2026-09-16T19:14:32.010599737Z ovnk_database_store/ovnkube-node-v2qjb_nbdb
2026-09-16T19:14:32.020732489Z ovnk_database_store/ovnkube-node-69rtg_nbdb
2026-09-16T19:14:32.035306133Z INFO: Network log collection complete.
2026-09-16T19:14:32.057196112Z Wrote inspect data to must-gather.
2026-09-16T19:14:32.221373278Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:32.224943953Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-16T19:14:32.228978799Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-16T19:14:32.317847965Z Gathering data for ns/openshift-console-operator...
2026-09-16T19:14:32.402762047Z Wrote inspect data to must-gather.
2026-09-16T19:14:32.479289807Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:32.658840774Z INFO: Getting status from alertmanager-main-0
2026-09-16T19:14:32.710748627Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:32.713094169Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-16T19:14:32.722860393Z Wrote inspect data to must-gather.
2026-09-16T19:14:32.826606616Z Gathering data for ns/openshift-console...
2026-09-16T19:14:32.940958622Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:33.023890976Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-16T19:14:33.026500638Z INFO: Fallback to identify the container image from release info
2026-09-16T19:14:33.146514579Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:33.149671363Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-16T19:14:33.378568143Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:33.379071361Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-16T19:14:33.579881960Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:33.582530822Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-16T19:14:33.680369234Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-16T19:14:33.778924878Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:33.832137848Z daemonset.apps/perf-node-gather-daemonset created
2026-09-16T19:14:33.986255572Z Waiting for performance profile collector pods to become ready: 1
2026-09-16T19:14:33.996334709Z Gathering data for ns/openshift-dns-operator...
2026-09-16T19:14:34.024385662Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:34.027945368Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-16T19:14:34.152689567Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-16T19:14:34.215384976Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:34.346344876Z Gathering data for ns/openshift-dns...
2026-09-16T19:14:34.421511669Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:34.424165103Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-16T19:14:34.596772234Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:34.798376416Z Wrote inspect data to /must-gather/istio.
2026-09-16T19:14:35.124702819Z Waiting for performance profile collector pods to become ready: 2
2026-09-16T19:14:35.153795918Z Gathering data for ns/openshift-image-registry...
2026-09-16T19:14:35.511217007Z Done executing Istio gather script
2026-09-16T19:14:35.659035822Z Gathering data for ns/openshift-ingress-operator...
2026-09-16T19:14:35.982322769Z Gathering data for ns/openshift-ingress...
2026-09-16T19:14:36.244345618Z Waiting for performance profile collector pods to become ready: 3
2026-09-16T19:14:36.327363372Z Gathering data for ns/openshift-ingress-canary...
2026-09-16T19:14:36.722571728Z Gathering data for ns/openshift-insights...
2026-09-16T19:14:37.378744683Z Waiting for performance profile collector pods to become ready: 4
2026-09-16T19:14:38.489616634Z Waiting for performance profile collector pods to become ready: 5
2026-09-16T19:14:39.170645487Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-16T19:14:39.171950123Z Gathering data for ns/kserve...
2026-09-16T19:14:39.613804118Z Waiting for performance profile collector pods to become ready: 6
2026-09-16T19:14:40.725639505Z Waiting for performance profile collector pods to become ready: 7
2026-09-16T19:14:40.808037847Z Gathering data for ns/openshift-config...
2026-09-16T19:14:41.079596317Z Gathering data for ns/openshift-config-managed...
2026-09-16T19:14:41.546398826Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-16T19:14:41.827167794Z Gathering data for ns/openshift-kube-apiserver...
2026-09-16T19:14:41.862782238Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-16T19:14:42.107964823Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-16T19:14:42.143256001Z Collecting performance related data for node ip-10-0-140-230.ec2.internal
2026-09-16T19:14:42.156416770Z Collecting performance related data for node ip-10-0-143-139.ec2.internal
2026-09-16T19:14:42.162256315Z Collecting performance related data for node ip-10-0-128-222.ec2.internal
2026-09-16T19:14:42.447756315Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-16T19:14:42.914435599Z Gathering data for ns/openshift-kube-scheduler...
2026-09-16T19:14:43.332238891Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-16T19:14:43.747833923Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-16T19:14:44.175056456Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-16T19:14:44.271069618Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-16T19:14:44.737608577Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-16T19:14:45.500275683Z Gathering data for ns/openshift-multus...
2026-09-16T19:14:47.693519117Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-16T19:14:48.969518680Z Gathering data for ns/openshift-host-network...
2026-09-16T19:14:49.181293156Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-16T19:14:49.298330692Z Gathering data for ns/openshift-network-diagnostics...
2026-09-16T19:14:49.872528001Z Gathering data for ns/openshift-network-node-identity...
2026-09-16T19:14:50.275450307Z Gathering data for ns/openshift-network-operator...
2026-09-16T19:14:50.637749603Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-16T19:14:50.660029111Z Collecting kubelet logs for node ip-10-0-140-230.ec2.internal
2026-09-16T19:14:50.662239923Z Collecting kubelet logs for node ip-10-0-143-139.ec2.internal
2026-09-16T19:14:50.664446817Z Collecting kubelet logs for node ip-10-0-128-222.ec2.internal
2026-09-16T19:14:50.970211085Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-16T19:14:51.051676156Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-68wb5 namespace
2026-09-16T19:14:51.056497382Z INFO: Node performance data collection complete.
2026-09-16T19:14:51.486963637Z Gathering data for ns/openshift-apiserver-operator...
2026-09-16T19:14:51.812336142Z Gathering data for ns/openshift-apiserver...
2026-09-16T19:14:52.193457430Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-16T19:14:52.463700155Z Gathering data for ns/openshift-controller-manager...
2026-09-16T19:14:52.780510643Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-16T19:14:53.432953387Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-16T19:14:53.761409013Z Gathering data for ns/openshift-service-ca-operator...
2026-09-16T19:14:54.088188885Z Gathering data for ns/openshift-service-ca...
2026-09-16T19:14:54.185723048Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-16T19:14:54.501916436Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-16T19:14:55.412813252Z Wrote inspect data to must-gather.
2026-09-16T19:14:55.412858720Z error: inspection completed with the errors occurred while gathering data:
2026-09-16T19:14:55.412858720Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-16T19:14:55.543981951Z error: the server doesn't have a resource type "clusters"
2026-09-16T19:14:55.671615266Z Caches written to disk