2026-07-28T16:48:58.579977239Z [disk usage checker] Started
2026-07-28T16:48:58.584299201Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-28T16:48:58.774848023Z Gathering data for ns/openshift-cluster-version...
2026-07-28T16:48:58.953213247Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T16:48:59.042550868Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T16:48:59.127396712Z Gathering data for ns/default...
2026-07-28T16:48:59.411623993Z Gathering data for ns/openshift...
2026-07-28T16:48:59.820315068Z Gathering data for ns/kube-system...
2026-07-28T16:49:00.467681507Z Gathering data for ns/openshift-etcd...
2026-07-28T16:49:00.764499892Z Wrote inspect data to must-gather.
2026-07-28T16:49:00.764526146Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T16:49:00.764526146Z     namespaces "assisted-installer" not found
2026-07-28T16:49:01.129815656Z Waiting on subprocesses to finish execution.
2026-07-28T16:49:01.140145005Z Executing Istio gather script
2026-07-28T16:49:01.141242322Z INFO: Gathering HAProxy config files
2026-07-28T16:49:01.148678144Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-28T16:49:01.151465708Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-28T16:49:01.152937282Z INFO: Collecting host service logs for crio
2026-07-28T16:49:01.153167882Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-28T16:49:01.153304450Z INFO: Collecting host service logs for kubelet
2026-07-28T16:49:01.153643770Z INFO: Collecting host service logs for rpm-ostreed
2026-07-28T16:49:01.153989524Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-28T16:49:01.154321550Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-28T16:49:01.154646189Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-28T16:49:01.154997182Z INFO: Collecting host service logs for NetworkManager
2026-07-28T16:49:01.155330403Z INFO: Collecting host service logs for openvswitch
2026-07-28T16:49:01.155655406Z INFO: Collecting host service logs for ovs-configuration
2026-07-28T16:49:01.155997120Z INFO: Collecting host service logs for ovsdb-server
2026-07-28T16:49:01.156329316Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-28T16:49:01.156721269Z INFO: Waiting for worker host service log collection to complete ...
2026-07-28T16:49:01.180477218Z INFO: Waiting for node performance related collection to complete ...
2026-07-28T16:49:01.506793554Z No resources found
2026-07-28T16:49:01.553089971Z error: the server doesn't have a resource type "performanceprofile"
2026-07-28T16:49:01.578546711Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-28T16:49:01.842231989Z No resources found in openshift-etcd namespace.
2026-07-28T16:49:01.902114971Z INFO: "metallb-operator" not detected. Skipping.
2026-07-28T16:49:01.997801659Z Wrote inspect data to must-gather.
2026-07-28T16:49:02.322738555Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-28T16:49:02.322823536Z INFO: on-disk MachineConfig config collection complete.
2026-07-28T16:49:02.374629656Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-28T16:49:02.476884508Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-v9pls
2026-07-28T16:49:02.477001225Z insights-runtime-extractor-62qjc
2026-07-28T16:49:02.477027821Z insights-runtime-extractor-dcjrq
2026-07-28T16:49:02.477049062Z insights-runtime-extractor-zmk5r
2026-07-28T16:49:02.516680692Z INFO: Worker host service log collection to complete.
2026-07-28T16:49:02.527506587Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-28T16:49:02.549526949Z No resources found
2026-07-28T16:49:02.576399474Z ERROR: No running kube-apiserver pods found
2026-07-28T16:49:02.750272464Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-28T16:49:02.759462785Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-28T16:49:02.814639765Z Wrote inspect data to must-gather.
2026-07-28T16:49:03.060173170Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-28T16:49:03.060173170Z See 'oc rsync -h' for help and examples
2026-07-28T16:49:03.187519358Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T16:49:03.258791225Z tar: Removing leading `/' from member names
2026-07-28T16:49:03.264375803Z INFO: HAProxy config collection complete.
2026-07-28T16:49:03.309454506Z Wrote inspect data to must-gather.
2026-07-28T16:49:03.342460081Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-28T16:49:03.364285619Z Wrote inspect data to must-gather.
2026-07-28T16:49:03.492745307Z Wrote inspect data to must-gather.
2026-07-28T16:49:03.517628818Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-28T16:49:03.534744191Z Wrote inspect data to must-gather.
2026-07-28T16:49:03.579072102Z Wrote inspect data to must-gather.
2026-07-28T16:49:03.592954344Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-28T16:49:03.636113500Z INFO: 'previous-logs' folder not found on ip-10-0-137-218.ec2.internal, skipping...
2026-07-28T16:49:03.644610108Z error: the server doesn't have a resource type "machineconfigs"
2026-07-28T16:49:03.791440450Z Gathering data for ns/openshift-monitoring...
2026-07-28T16:49:03.826304235Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-28T16:49:03.843170231Z Wrote inspect data to must-gather.
2026-07-28T16:49:03.930582997Z INFO: 'previous-logs' folder not found on ip-10-0-140-117.ec2.internal, skipping...
2026-07-28T16:49:03.999557069Z INFO: Getting rules from prometheus-k8s-0
2026-07-28T16:49:04.113180489Z Wrote inspect data to must-gather.
2026-07-28T16:49:04.146959142Z Wrote inspect data to must-gather.
2026-07-28T16:49:04.185275453Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-28T16:49:04.249779879Z INFO: 'previous-logs' folder not found on ip-10-0-140-245.ec2.internal, skipping...
2026-07-28T16:49:04.249813811Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-28T16:49:04.249813811Z INFO: Machine Config Daemon termination log collection complete.
2026-07-28T16:49:04.278201085Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-28T16:49:04.398163686Z Wrote inspect data to must-gather.
2026-07-28T16:49:04.440633982Z INFO: Getting status/config from prometheus-k8s-0
2026-07-28T16:49:04.490178067Z Wrote inspect data to must-gather.
2026-07-28T16:49:04.674111134Z Wrote inspect data to must-gather.
2026-07-28T16:49:04.684542689Z INFO: INTERCONNECT MODE
2026-07-28T16:49:04.684568818Z INFO: Gathering ovn-kubernetes DBs
2026-07-28T16:49:04.838099340Z INFO: Gathering OVN_Northbound from ovnkube-node-5rc9g...
2026-07-28T16:49:04.838369573Z INFO: Gathering OVN_Northbound from ovnkube-node-lxr6b...
2026-07-28T16:49:04.838659484Z INFO: Gathering OVN_Northbound from ovnkube-node-vjvs7...
2026-07-28T16:49:04.838992873Z INFO: Gathering OVN_Southbound from ovnkube-node-5rc9g...
2026-07-28T16:49:04.839308484Z INFO: Gathering OVN_Southbound from ovnkube-node-lxr6b...
2026-07-28T16:49:04.839585586Z INFO: Gathering OVN_Southbound from ovnkube-node-vjvs7...
2026-07-28T16:49:04.853513835Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-28T16:49:04.915970931Z Wrote inspect data to must-gather.
2026-07-28T16:49:04.921702825Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-28T16:49:05.148951874Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:05.174987880Z tar: Removing leading `/' from member names
2026-07-28T16:49:05.186341331Z tar: Removing leading `/' from member names
2026-07-28T16:49:05.193346244Z tar: Removing leading `/' from member names
2026-07-28T16:49:05.211147202Z tar: Removing leading `/' from member names
2026-07-28T16:49:05.222247594Z tar: Removing leading `/' from member names
2026-07-28T16:49:05.240745443Z tar: Removing leading `/' from member names
2026-07-28T16:49:05.268758317Z Wrote inspect data to must-gather.
2026-07-28T16:49:05.291789835Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-28T16:49:05.365371148Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:05.368002639Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-28T16:49:05.506397915Z Wrote inspect data to must-gather.
2026-07-28T16:49:05.575105000Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-28T16:49:05.589434841Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:05.657078662Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T16:49:05.663131321Z INFO: Waiting for network log collection to complete ...
2026-07-28T16:49:05.663281718Z INFO: Waiting for ovnk database copies to complete ...
2026-07-28T16:49:05.663965807Z INFO: Copying ovnk databases complete.
2026-07-28T16:49:05.666470486Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-07-28T16:49:05.670609885Z ovnk_database_store/
2026-07-28T16:49:05.670609885Z ovnk_database_store/ovnkube-node-5rc9g_nbdb
2026-07-28T16:49:05.677668240Z ovnk_database_store/ovnkube-node-5rc9g_sbdb
2026-07-28T16:49:05.698664385Z ovnk_database_store/ovnkube-node-vjvs7_sbdb
2026-07-28T16:49:05.717680951Z ovnk_database_store/ovnkube-node-lxr6b_nbdb
2026-07-28T16:49:05.724693934Z ovnk_database_store/ovnkube-node-lxr6b_sbdb
2026-07-28T16:49:05.739849159Z Wrote inspect data to must-gather.
2026-07-28T16:49:05.747542501Z ovnk_database_store/ovnkube-node-vjvs7_nbdb
2026-07-28T16:49:05.757470235Z INFO: Network log collection complete.
2026-07-28T16:49:05.802590988Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:05.805884135Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-28T16:49:05.870188450Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-28T16:49:05.932852230Z Wrote inspect data to must-gather.
2026-07-28T16:49:06.014636209Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:06.192907018Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-28T16:49:06.195432689Z INFO: Fallback to identify the container image from release info
2026-07-28T16:49:06.216269412Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:06.218884403Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-28T16:49:06.254578861Z INFO: Getting status from alertmanager-main-0
2026-07-28T16:49:06.258384241Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T16:49:06.438751472Z Gathering data for ns/openshift-network-console...
2026-07-28T16:49:06.447142460Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:06.627709382Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:06.631878951Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-28T16:49:06.797614809Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:06.828541921Z Gathering data for ns/openshift-console-operator...
2026-07-28T16:49:06.979313440Z Wrote inspect data to /must-gather/istio.
2026-07-28T16:49:07.146663371Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-28T16:49:07.205093329Z Gathering data for ns/openshift-console...
2026-07-28T16:49:07.265945905Z daemonset.apps/perf-node-gather-daemonset created
2026-07-28T16:49:07.409943980Z Waiting for performance profile collector pods to become ready: 1
2026-07-28T16:49:07.645749488Z Done executing Istio gather script
2026-07-28T16:49:07.759341844Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-28T16:49:08.228830003Z Gathering data for ns/openshift-dns-operator...
2026-07-28T16:49:08.488464884Z Gathering data for ns/openshift-dns...
2026-07-28T16:49:08.518765291Z Waiting for performance profile collector pods to become ready: 2
2026-07-28T16:49:08.598444606Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-28T16:49:09.049533800Z Gathering data for ns/openshift-image-registry...
2026-07-28T16:49:09.483782354Z Gathering data for ns/openshift-ingress-operator...
2026-07-28T16:49:09.628954248Z Waiting for performance profile collector pods to become ready: 3
2026-07-28T16:49:09.791966993Z Gathering data for ns/openshift-ingress...
2026-07-28T16:49:10.096216570Z Gathering data for ns/openshift-ingress-canary...
2026-07-28T16:49:10.471233670Z Gathering data for ns/openshift-insights...
2026-07-28T16:49:10.729327896Z Waiting for performance profile collector pods to become ready: 4
2026-07-28T16:49:11.829265115Z Waiting for performance profile collector pods to become ready: 5
2026-07-28T16:49:12.253587166Z Gathering data for ns/openshift-jobset-operator...
2026-07-28T16:49:12.944143166Z Waiting for performance profile collector pods to become ready: 6
2026-07-28T16:49:13.033508329Z Gathering data for ns/openshift-config...
2026-07-28T16:49:13.276727406Z Gathering data for ns/openshift-config-managed...
2026-07-28T16:49:13.603199327Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-28T16:49:13.717905241Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-28T16:49:13.957391913Z Gathering data for ns/openshift-kube-apiserver...
2026-07-28T16:49:14.062499236Z Waiting for performance profile collector pods to become ready: 7
2026-07-28T16:49:14.201224116Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-28T16:49:14.505889477Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-28T16:49:14.784337229Z Gathering data for ns/openshift-kube-scheduler...
2026-07-28T16:49:15.018529012Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-28T16:49:15.165524251Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-28T16:49:15.306092564Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-28T16:49:15.423319433Z Collecting performance related data for node ip-10-0-140-245.ec2.internal
2026-07-28T16:49:15.430122688Z Collecting performance related data for node ip-10-0-137-218.ec2.internal
2026-07-28T16:49:15.437334217Z Collecting performance related data for node ip-10-0-140-117.ec2.internal
2026-07-28T16:49:15.653421211Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-28T16:49:16.000029243Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-28T16:49:16.617389659Z Gathering data for ns/openshift-multus...
2026-07-28T16:49:18.072299745Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-28T16:49:18.607995612Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-28T16:49:19.275361431Z Gathering data for ns/openshift-host-network...
2026-07-28T16:49:19.631011350Z Gathering data for ns/openshift-network-diagnostics...
2026-07-28T16:49:20.169996602Z Gathering data for ns/openshift-network-node-identity...
2026-07-28T16:49:20.623550879Z Gathering data for ns/openshift-network-operator...
2026-07-28T16:49:20.990236748Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-28T16:49:21.261733691Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-28T16:49:21.715625678Z Gathering data for ns/openshift-apiserver-operator...
2026-07-28T16:49:21.991424789Z Gathering data for ns/openshift-apiserver...
2026-07-28T16:49:22.365186822Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-28T16:49:22.625341922Z Gathering data for ns/openshift-controller-manager...
2026-07-28T16:49:22.895358615Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-28T16:49:23.411661057Z Collecting kubelet logs for node ip-10-0-140-245.ec2.internal
2026-07-28T16:49:23.413383480Z Collecting kubelet logs for node ip-10-0-137-218.ec2.internal
2026-07-28T16:49:23.415290771Z Collecting kubelet logs for node ip-10-0-140-117.ec2.internal
2026-07-28T16:49:23.591494367Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-28T16:49:23.614723956Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-28T16:49:23.738108486Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ndr5k namespace
2026-07-28T16:49:23.743953389Z INFO: Node performance data collection complete.
2026-07-28T16:49:23.875380861Z Gathering data for ns/openshift-service-ca-operator...
2026-07-28T16:49:24.174126358Z Gathering data for ns/openshift-service-ca...
2026-07-28T16:49:24.505614781Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-28T16:49:25.354692689Z Wrote inspect data to must-gather.
2026-07-28T16:49:25.354741498Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T16:49:25.354741498Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-28T16:49:25.469889114Z error: the server doesn't have a resource type "clusters"
2026-07-28T16:49:25.608570631Z Caches written to disk