2026-09-17T16:39:37.306008798Z [disk usage checker] Started
2026-09-17T16:39:37.309352201Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-17T16:39:37.522028412Z Gathering data for ns/openshift-cluster-version...
2026-09-17T16:39:37.670907914Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T16:39:37.764598515Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:39:37.850398856Z Gathering data for ns/default...
2026-09-17T16:39:38.157116015Z Gathering data for ns/openshift...
2026-09-17T16:39:38.617117225Z Gathering data for ns/kube-system...
2026-09-17T16:39:39.371751684Z Gathering data for ns/openshift-etcd...
2026-09-17T16:39:39.698023450Z Wrote inspect data to must-gather.
2026-09-17T16:39:39.698053621Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T16:39:39.698053621Z     namespaces "assisted-installer" not found
2026-09-17T16:39:40.082452344Z Waiting on subprocesses to finish execution.
2026-09-17T16:39:40.089453723Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T16:39:40.090835733Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T16:39:40.100822928Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T16:39:40.103653378Z Executing Istio gather script
2026-09-17T16:39:40.106041122Z INFO: Gathering HAProxy config files
2026-09-17T16:39:40.114202027Z INFO: Collecting host service logs for crio
2026-09-17T16:39:40.114550950Z INFO: Collecting host service logs for kubelet
2026-09-17T16:39:40.114893050Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T16:39:40.115219628Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T16:39:40.115451505Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T16:39:40.115581972Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T16:39:40.115919287Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T16:39:40.116262934Z INFO: Collecting host service logs for NetworkManager
2026-09-17T16:39:40.116593739Z INFO: Collecting host service logs for openvswitch
2026-09-17T16:39:40.116939209Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T16:39:40.117301109Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T16:39:40.117634936Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T16:39:40.118081590Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T16:39:40.830024260Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T16:39:41.095139104Z Wrote inspect data to must-gather.
2026-09-17T16:39:41.188529384Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T16:39:41.200141858Z ERROR: No running kube-apiserver pods found
2026-09-17T16:39:41.311636927Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T16:39:41.507067745Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T16:39:41.552430442Z No resources found
2026-09-17T16:39:41.575670790Z No resources found in openshift-etcd namespace.
2026-09-17T16:39:41.582339076Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T16:39:41.607401303Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T16:39:41.607517429Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T16:39:41.663710071Z Wrote inspect data to must-gather.
2026-09-17T16:39:41.697819124Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-wv9sd
2026-09-17T16:39:41.697866868Z insights-runtime-extractor-7hpwm
2026-09-17T16:39:41.697885758Z insights-runtime-extractor-k79n7
2026-09-17T16:39:41.697904921Z insights-runtime-extractor-nf8md
2026-09-17T16:39:41.723410607Z INFO: Worker host service log collection to complete.
2026-09-17T16:39:41.758138697Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T16:39:41.778189553Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T16:39:41.965839485Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T16:39:41.983936633Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T16:39:41.983936633Z See 'oc rsync -h' for help and examples
2026-09-17T16:39:42.239941715Z Wrote inspect data to must-gather.
2026-09-17T16:39:42.348294657Z No resources found
2026-09-17T16:39:42.352322027Z Wrote inspect data to must-gather.
2026-09-17T16:39:42.402577911Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-17T16:39:42.420081499Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:39:42.507641757Z Gathering data for ns/openshift-monitoring...
2026-09-17T16:39:42.554085434Z Wrote inspect data to must-gather.
2026-09-17T16:39:42.594318393Z INFO: 'previous-logs' folder not found on ip-10-0-131-222.ec2.internal, skipping...
2026-09-17T16:39:42.597537327Z tar: Removing leading `/' from member names
2026-09-17T16:39:42.609531123Z INFO: HAProxy config collection complete.
2026-09-17T16:39:42.640997966Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T16:39:42.677666764Z Wrote inspect data to must-gather.
2026-09-17T16:39:42.734344027Z Wrote inspect data to must-gather.
2026-09-17T16:39:42.918945596Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T16:39:42.941005129Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T16:39:42.953287965Z Wrote inspect data to must-gather.
2026-09-17T16:39:43.002397620Z INFO: 'previous-logs' folder not found on ip-10-0-139-63.ec2.internal, skipping...
2026-09-17T16:39:43.118795515Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T16:39:43.246756485Z Wrote inspect data to must-gather.
2026-09-17T16:39:43.318487791Z INFO: 'previous-logs' folder not found on ip-10-0-142-173.ec2.internal, skipping...
2026-09-17T16:39:43.318524006Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T16:39:43.318565902Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T16:39:43.396469478Z Wrote inspect data to must-gather.
2026-09-17T16:39:43.447623306Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T16:39:43.532655850Z Wrote inspect data to must-gather.
2026-09-17T16:39:43.590356160Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T16:39:43.744352773Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T16:39:43.843132765Z Wrote inspect data to must-gather.
2026-09-17T16:39:43.876494541Z INFO: INTERCONNECT MODE
2026-09-17T16:39:43.876583456Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T16:39:43.909453411Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T16:39:44.048917332Z INFO: Gathering OVN_Northbound from ovnkube-node-9q7bk...
2026-09-17T16:39:44.049122665Z INFO: Gathering OVN_Northbound from ovnkube-node-fhf6q...
2026-09-17T16:39:44.049318935Z INFO: Gathering OVN_Northbound from ovnkube-node-xxcmj...
2026-09-17T16:39:44.049558743Z INFO: Gathering OVN_Southbound from ovnkube-node-9q7bk...
2026-09-17T16:39:44.049826330Z INFO: Gathering OVN_Southbound from ovnkube-node-fhf6q...
2026-09-17T16:39:44.050071928Z INFO: Gathering OVN_Southbound from ovnkube-node-xxcmj...
2026-09-17T16:39:44.098556874Z Wrote inspect data to must-gather.
2026-09-17T16:39:44.174929754Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T16:39:44.385981034Z tar: Removing leading `/' from member names
2026-09-17T16:39:44.401861310Z tar: Removing leading `/' from member names
2026-09-17T16:39:44.424576751Z tar: Removing leading `/' from member names
2026-09-17T16:39:44.432663395Z tar: Removing leading `/' from member names
2026-09-17T16:39:44.442984354Z tar: Removing leading `/' from member names
2026-09-17T16:39:44.443186025Z tar: Removing leading `/' from member names
2026-09-17T16:39:44.476903255Z Wrote inspect data to must-gather.
2026-09-17T16:39:44.547890089Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T16:39:44.573343647Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T16:39:44.747975121Z Wrote inspect data to must-gather.
2026-09-17T16:39:44.869216714Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T16:39:44.963422643Z INFO: Waiting for network log collection to complete ...
2026-09-17T16:39:44.963472972Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T16:39:44.963976934Z INFO: Copying ovnk databases complete.
2026-09-17T16:39:44.965966534Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-09-17T16:39:44.968243040Z ovnk_database_store/
2026-09-17T16:39:44.968243040Z ovnk_database_store/ovnkube-node-fhf6q_nbdb
2026-09-17T16:39:44.976927374Z ovnk_database_store/ovnkube-node-9q7bk_sbdb
2026-09-17T16:39:44.997686453Z ovnk_database_store/ovnkube-node-9q7bk_nbdb
2026-09-17T16:39:45.005845765Z ovnk_database_store/ovnkube-node-fhf6q_sbdb
2026-09-17T16:39:45.009172584Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T16:39:45.010051482Z Wrote inspect data to must-gather.
2026-09-17T16:39:45.027177500Z ovnk_database_store/ovnkube-node-xxcmj_nbdb
2026-09-17T16:39:45.034756449Z ovnk_database_store/ovnkube-node-xxcmj_sbdb
2026-09-17T16:39:45.054973616Z INFO: Network log collection complete.
2026-09-17T16:39:45.190876773Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:39:45.217681064Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:45.316823928Z INFO: Getting status from alertmanager-main-0
2026-09-17T16:39:45.323917762Z Wrote inspect data to must-gather.
2026-09-17T16:39:45.419286098Z Gathering data for ns/openshift-network-console...
2026-09-17T16:39:45.461751752Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:45.466625264Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T16:39:45.627408608Z Wrote inspect data to must-gather.
2026-09-17T16:39:45.674881286Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:45.913479014Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:45.916747271Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T16:39:45.919431652Z Gathering data for ns/openshift-console-operator...
2026-09-17T16:39:45.934977568Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T16:39:45.936706680Z INFO: Fallback to identify the container image from release info
2026-09-17T16:39:46.125946535Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:46.324245807Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:46.327280494Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T16:39:46.329907499Z Gathering data for ns/openshift-console...
2026-09-17T16:39:46.555861466Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:46.558981261Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-17T16:39:46.696427205Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T16:39:46.779478631Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:46.782299892Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-17T16:39:46.858641337Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T16:39:46.930001694Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T16:39:46.975588798Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:47.208371090Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:47.211169522Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-17T16:39:47.411172742Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-17T16:39:47.417293595Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:47.559061972Z Gathering data for ns/openshift-dns-operator...
2026-09-17T16:39:47.653268419Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:47.655650528Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T16:39:47.814998736Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:47.952759420Z Gathering data for ns/openshift-dns...
2026-09-17T16:39:47.994597662Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T16:39:48.022152695Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:39:48.693443392Z Gathering data for ns/openshift-image-registry...
2026-09-17T16:39:48.711346369Z Done executing Istio gather script
2026-09-17T16:39:49.117919773Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T16:39:49.226622400Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T16:39:49.618528680Z Gathering data for ns/openshift-ingress...
2026-09-17T16:39:50.020826884Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T16:39:50.245411327Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T16:39:50.442588077Z Gathering data for ns/openshift-insights...
2026-09-17T16:39:51.366913077Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T16:39:52.416908629Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-17T16:39:52.483095667Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T16:39:52.864980088Z Gathering data for ns/kserve...
2026-09-17T16:39:53.618574468Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T16:39:54.406070758Z Gathering data for ns/openshift-config...
2026-09-17T16:39:54.751881861Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T16:39:54.755312502Z Gathering data for ns/openshift-config-managed...
2026-09-17T16:39:55.012202611Z Collecting performance related data for node ip-10-0-139-63.ec2.internal
2026-09-17T16:39:55.021882252Z Collecting performance related data for node ip-10-0-131-222.ec2.internal
2026-09-17T16:39:55.030357564Z Collecting performance related data for node ip-10-0-142-173.ec2.internal
2026-09-17T16:39:55.288944285Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T16:39:55.634309358Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T16:39:56.011359803Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T16:39:56.420814013Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T16:39:56.785226588Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T16:39:57.209368906Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T16:39:57.426257220Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-17T16:39:57.636351527Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T16:39:58.046043917Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T16:39:58.468546966Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T16:39:59.036116482Z Gathering data for ns/openshift-multus...
2026-09-17T16:40:00.706719594Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T16:40:02.077951209Z Gathering data for ns/openshift-host-network...
2026-09-17T16:40:02.412481141Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T16:40:02.431538005Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-17T16:40:03.063900215Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T16:40:03.349262699Z Collecting kubelet logs for node ip-10-0-139-63.ec2.internal
2026-09-17T16:40:03.350983887Z Collecting kubelet logs for node ip-10-0-131-222.ec2.internal
2026-09-17T16:40:03.352651680Z Collecting kubelet logs for node ip-10-0-142-173.ec2.internal
2026-09-17T16:40:03.577743809Z Gathering data for ns/openshift-network-operator...
2026-09-17T16:40:03.701927288Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-tf9tl namespace
2026-09-17T16:40:03.706816568Z INFO: Node performance data collection complete.
2026-09-17T16:40:04.064139325Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T16:40:04.383541451Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T16:40:04.948462384Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T16:40:05.289988385Z Gathering data for ns/openshift-apiserver...
2026-09-17T16:40:05.696030186Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T16:40:05.970349018Z Gathering data for ns/openshift-controller-manager...
2026-09-17T16:40:06.290823685Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T16:40:06.938926764Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T16:40:07.285140432Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T16:40:07.435902787Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-09-17T16:40:07.609813904Z Gathering data for ns/openshift-service-ca...
2026-09-17T16:40:07.989762367Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T16:40:08.930488039Z Wrote inspect data to must-gather.
2026-09-17T16:40:08.930529140Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T16:40:08.930529140Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T16:40:09.058760873Z error: the server doesn't have a resource type "clusters"
2026-09-17T16:40:09.196640799Z Caches written to disk