2026-09-19T08:49:08.553860728Z [disk usage checker] Started
2026-09-19T08:49:08.557313660Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-19T08:49:08.750594493Z Gathering data for ns/openshift-cluster-version...
2026-09-19T08:49:08.879719682Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T08:49:08.967362508Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T08:49:09.047185586Z Gathering data for ns/default...
2026-09-19T08:49:09.391746829Z Gathering data for ns/openshift...
2026-09-19T08:49:09.812609358Z Gathering data for ns/kube-system...
2026-09-19T08:49:10.515531155Z Gathering data for ns/openshift-etcd...
2026-09-19T08:49:10.812338928Z Wrote inspect data to must-gather.
2026-09-19T08:49:10.812372969Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T08:49:10.812372969Z     namespaces "assisted-installer" not found
2026-09-19T08:49:11.382471643Z Waiting on subprocesses to finish execution.
2026-09-19T08:49:11.385815030Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-19T08:49:11.393013645Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-19T08:49:11.396865736Z Executing Istio gather script
2026-09-19T08:49:11.408453999Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-19T08:49:11.412571320Z INFO: Gathering HAProxy config files
2026-09-19T08:49:11.423406512Z INFO: Collecting host service logs for crio
2026-09-19T08:49:11.423746033Z INFO: Collecting host service logs for kubelet
2026-09-19T08:49:11.424078295Z INFO: Collecting host service logs for rpm-ostreed
2026-09-19T08:49:11.424413174Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-19T08:49:11.424729871Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-19T08:49:11.425040273Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-19T08:49:11.425358990Z INFO: Collecting host service logs for NetworkManager
2026-09-19T08:49:11.425672813Z INFO: Collecting host service logs for openvswitch
2026-09-19T08:49:11.425965155Z INFO: Collecting host service logs for ovs-configuration
2026-09-19T08:49:11.426302553Z INFO: Collecting host service logs for ovsdb-server
2026-09-19T08:49:11.426630516Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-19T08:49:11.427008636Z INFO: Waiting for worker host service log collection to complete ...
2026-09-19T08:49:11.477813472Z INFO: Waiting for node performance related collection to complete ...
2026-09-19T08:49:11.899300834Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-19T08:49:12.242100113Z No resources found
2026-09-19T08:49:12.265767803Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-19T08:49:12.405962596Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-19T08:49:12.546736015Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-hhscx
2026-09-19T08:49:12.546736015Z insights-runtime-extractor-jvd6q
2026-09-19T08:49:12.546736015Z insights-runtime-extractor-t7vcg
2026-09-19T08:49:12.546736015Z insights-runtime-extractor-vhgvm
2026-09-19T08:49:12.577397130Z INFO: "metallb-operator" not detected. Skipping.
2026-09-19T08:49:12.640471955Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-19T08:49:12.805607180Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-19T08:49:12.805653850Z INFO: on-disk MachineConfig config collection complete.
2026-09-19T08:49:12.815127661Z Wrote inspect data to must-gather.
2026-09-19T08:49:12.865969164Z No resources found in openshift-etcd namespace.
2026-09-19T08:49:13.055833100Z Wrote inspect data to must-gather.
2026-09-19T08:49:13.245720612Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-19T08:49:13.361370712Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T08:49:13.397403941Z INFO: Worker host service log collection to complete.
2026-09-19T08:49:13.619983808Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-19T08:49:13.668567100Z error: the server doesn't have a resource type "performanceprofile"
2026-09-19T08:49:13.677895871Z ERROR: No running kube-apiserver pods found
2026-09-19T08:49:13.706224333Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-19T08:49:13.798764055Z No resources found
2026-09-19T08:49:13.892445095Z Wrote inspect data to must-gather.
2026-09-19T08:49:13.892618922Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-19T08:49:13.892618922Z See 'oc rsync -h' for help and examples
2026-09-19T08:49:13.895564882Z Wrote inspect data to must-gather.
2026-09-19T08:49:13.927577500Z INFO: 'previous-logs' folder not found on ip-10-0-134-26.ec2.internal, skipping...
2026-09-19T08:49:13.936197123Z Wrote inspect data to must-gather.
2026-09-19T08:49:13.969030412Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-19T08:49:13.989568211Z Wrote inspect data to must-gather.
2026-09-19T08:49:14.205077139Z Wrote inspect data to must-gather.
2026-09-19T08:49:14.262030135Z tar: Removing leading `/' from member names
2026-09-19T08:49:14.267095037Z INFO: HAProxy config collection complete.
2026-09-19T08:49:14.272298795Z Gathering data for ns/openshift-monitoring...
2026-09-19T08:49:14.318596170Z INFO: Getting rules from prometheus-k8s-0
2026-09-19T08:49:14.340867823Z INFO: 'previous-logs' folder not found on ip-10-0-140-141.ec2.internal, skipping...
2026-09-19T08:49:14.395589548Z Wrote inspect data to must-gather.
2026-09-19T08:49:14.480327296Z error: the server doesn't have a resource type "machineconfigs"
2026-09-19T08:49:14.663156650Z INFO: 'previous-logs' folder not found on ip-10-0-142-103.ec2.internal, skipping...
2026-09-19T08:49:14.663202940Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-19T08:49:14.663215583Z INFO: Machine Config Daemon termination log collection complete.
2026-09-19T08:49:14.676234165Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-19T08:49:14.695247079Z Wrote inspect data to must-gather.
2026-09-19T08:49:14.847598988Z INFO: Getting status/config from prometheus-k8s-0
2026-09-19T08:49:14.960648867Z Wrote inspect data to must-gather.
2026-09-19T08:49:15.037405042Z Wrote inspect data to must-gather.
2026-09-19T08:49:15.072864447Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-19T08:49:15.150819238Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-19T08:49:15.305219455Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-19T08:49:15.352749805Z INFO: INTERCONNECT MODE
2026-09-19T08:49:15.352845140Z INFO: Gathering ovn-kubernetes DBs
2026-09-19T08:49:15.390693925Z Wrote inspect data to must-gather.
2026-09-19T08:49:15.531272666Z INFO: Gathering OVN_Northbound from ovnkube-node-bkpds...
2026-09-19T08:49:15.531485429Z INFO: Gathering OVN_Northbound from ovnkube-node-hb59d...
2026-09-19T08:49:15.531687086Z INFO: Gathering OVN_Northbound from ovnkube-node-hnft5...
2026-09-19T08:49:15.531948762Z INFO: Gathering OVN_Southbound from ovnkube-node-bkpds...
2026-09-19T08:49:15.532367620Z INFO: Gathering OVN_Southbound from ovnkube-node-hb59d...
2026-09-19T08:49:15.532704156Z INFO: Gathering OVN_Southbound from ovnkube-node-hnft5...
2026-09-19T08:49:15.568924365Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-19T08:49:15.672037428Z Wrote inspect data to must-gather.
2026-09-19T08:49:15.821395633Z tar: Removing leading `/' from member names
2026-09-19T08:49:15.871471669Z tar: Removing leading `/' from member names
2026-09-19T08:49:15.879389838Z tar: Removing leading `/' from member names
2026-09-19T08:49:15.909853131Z tar: Removing leading `/' from member names
2026-09-19T08:49:15.921366453Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-19T08:49:15.926415219Z tar: Removing leading `/' from member names
2026-09-19T08:49:15.931971931Z tar: Removing leading `/' from member names
2026-09-19T08:49:15.971478044Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T08:49:16.073756069Z Wrote inspect data to must-gather.
2026-09-19T08:49:16.192885536Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-19T08:49:16.345037607Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-19T08:49:16.358817404Z Wrote inspect data to must-gather.
2026-09-19T08:49:16.510354603Z INFO: Waiting for network log collection to complete ...
2026-09-19T08:49:16.510403953Z INFO: Waiting for ovnk database copies to complete ...
2026-09-19T08:49:16.510923189Z INFO: Copying ovnk databases complete.
2026-09-19T08:49:16.512610174Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-09-19T08:49:16.514885338Z ovnk_database_store/
2026-09-19T08:49:16.514885338Z ovnk_database_store/ovnkube-node-hb59d_nbdb
2026-09-19T08:49:16.524003958Z ovnk_database_store/ovnkube-node-bkpds_nbdb
2026-09-19T08:49:16.537461688Z ovnk_database_store/ovnkube-node-hnft5_nbdb
2026-09-19T08:49:16.547461407Z ovnk_database_store/ovnkube-node-hnft5_sbdb
2026-09-19T08:49:16.554839975Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:16.574607811Z ovnk_database_store/ovnkube-node-hb59d_sbdb
2026-09-19T08:49:16.585213998Z W0919 08:49:16.585174    1464 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-19T08:49:16.590364673Z Wrote inspect data to must-gather.
2026-09-19T08:49:16.598817596Z ovnk_database_store/ovnkube-node-bkpds_sbdb
2026-09-19T08:49:16.629298170Z INFO: Network log collection complete.
2026-09-19T08:49:16.648646809Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T08:49:16.662596216Z INFO: Getting status from alertmanager-main-0
2026-09-19T08:49:16.828035814Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:16.830304604Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-19T08:49:16.866618301Z Gathering data for ns/openshift-network-console...
2026-09-19T08:49:16.937162647Z Wrote inspect data to must-gather.
2026-09-19T08:49:17.049285664Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:17.255643850Z Wrote inspect data to must-gather.
2026-09-19T08:49:17.278187708Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:17.281356364Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-19T08:49:17.343596345Z Gathering data for ns/openshift-console-operator...
2026-09-19T08:49:17.501507263Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:17.557443256Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-19T08:49:17.560157202Z INFO: Fallback to identify the container image from release info
2026-09-19T08:49:17.722477233Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:17.725223123Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-19T08:49:17.759645036Z Gathering data for ns/openshift-console...
2026-09-19T08:49:17.946973727Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:18.145680472Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-19T08:49:18.161331232Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:18.164615336Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-19T08:49:18.299219197Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-19T08:49:18.303230435Z daemonset.apps/perf-node-gather-daemonset created
2026-09-19T08:49:18.373038843Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:18.458530887Z Waiting for performance profile collector pods to become ready: 1
2026-09-19T08:49:18.586209218Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:18.588597924Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-19T08:49:18.747119511Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-19T08:49:18.795028371Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:18.867278311Z Gathering data for ns/openshift-dns-operator...
2026-09-19T08:49:19.012165931Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:19.014597595Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-19T08:49:19.192279476Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:19.209270760Z Gathering data for ns/openshift-dns...
2026-09-19T08:49:19.378453251Z Wrote inspect data to /must-gather/istio.
2026-09-19T08:49:19.600005105Z Waiting for performance profile collector pods to become ready: 2
2026-09-19T08:49:19.829829687Z Gathering data for ns/openshift-image-registry...
2026-09-19T08:49:20.097885739Z Done executing Istio gather script
2026-09-19T08:49:20.289862961Z Gathering data for ns/openshift-ingress-operator...
2026-09-19T08:49:20.594885973Z Gathering data for ns/openshift-ingress...
2026-09-19T08:49:20.737665228Z Waiting for performance profile collector pods to become ready: 3
2026-09-19T08:49:20.926518536Z Gathering data for ns/openshift-ingress-canary...
2026-09-19T08:49:21.313396087Z Gathering data for ns/openshift-insights...
2026-09-19T08:49:21.865839818Z Waiting for performance profile collector pods to become ready: 4
2026-09-19T08:49:22.983927484Z Waiting for performance profile collector pods to become ready: 5
2026-09-19T08:49:23.678619361Z Gathering data for ns/kserve...
2026-09-19T08:49:23.753054594Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-19T08:49:24.116489217Z Waiting for performance profile collector pods to become ready: 6
2026-09-19T08:49:25.234879902Z Waiting for performance profile collector pods to become ready: 7
2026-09-19T08:49:25.441201476Z Gathering data for ns/openshift-config...
2026-09-19T08:49:25.698227100Z Gathering data for ns/openshift-config-managed...
2026-09-19T08:49:26.090805524Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-19T08:49:26.372924566Z Waiting for performance profile collector pods to become ready: 8
2026-09-19T08:49:26.395608361Z Gathering data for ns/openshift-kube-apiserver...
2026-09-19T08:49:26.652653807Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-19T08:49:26.912910802Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-19T08:49:27.187169437Z Gathering data for ns/openshift-kube-scheduler...
2026-09-19T08:49:27.436379790Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-19T08:49:27.521867862Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-19T08:49:27.747487149Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-19T08:49:27.785865980Z Collecting performance related data for node ip-10-0-142-103.ec2.internal
2026-09-19T08:49:27.793068590Z Collecting performance related data for node ip-10-0-134-26.ec2.internal
2026-09-19T08:49:27.801357908Z Collecting performance related data for node ip-10-0-140-141.ec2.internal
2026-09-19T08:49:28.123946556Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-19T08:49:28.501609696Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-19T08:49:28.762152154Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-19T08:49:29.100869986Z Gathering data for ns/openshift-multus...
2026-09-19T08:49:30.561852383Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-19T08:49:31.754287440Z Gathering data for ns/openshift-host-network...
2026-09-19T08:49:32.041983250Z Gathering data for ns/openshift-network-diagnostics...
2026-09-19T08:49:32.555200763Z Gathering data for ns/openshift-network-node-identity...
2026-09-19T08:49:32.946812152Z Gathering data for ns/openshift-network-operator...
2026-09-19T08:49:33.312285250Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-19T08:49:33.577344674Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-19T08:49:33.767005430Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-09-19T08:49:34.028952594Z Gathering data for ns/openshift-apiserver-operator...
2026-09-19T08:49:34.346925669Z Gathering data for ns/openshift-apiserver...
2026-09-19T08:49:34.723668745Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-19T08:49:35.020766577Z Gathering data for ns/openshift-controller-manager...
2026-09-19T08:49:35.278471013Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-19T08:49:35.907612697Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-19T08:49:36.240467706Z Gathering data for ns/openshift-service-ca-operator...
2026-09-19T08:49:36.253198285Z Collecting kubelet logs for node ip-10-0-142-103.ec2.internal
2026-09-19T08:49:36.254975168Z Collecting kubelet logs for node ip-10-0-134-26.ec2.internal
2026-09-19T08:49:36.256795808Z Collecting kubelet logs for node ip-10-0-140-141.ec2.internal
2026-09-19T08:49:36.617925532Z Gathering data for ns/openshift-service-ca...
2026-09-19T08:49:36.715627231Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-lnnkg namespace
2026-09-19T08:49:36.720098895Z INFO: Node performance data collection complete.
2026-09-19T08:49:37.002222860Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-19T08:49:37.870870995Z Wrote inspect data to must-gather.
2026-09-19T08:49:37.870918378Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T08:49:37.870918378Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-19T08:49:38.003856117Z error: the server doesn't have a resource type "clusters"
2026-09-19T08:49:38.148994324Z Caches written to disk