2026-07-28T10:44:17.642102292Z [disk usage checker] Started
2026-07-28T10:44:17.645813710Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-28T10:44:17.845353428Z Gathering data for ns/openshift-cluster-version...
2026-07-28T10:44:17.979031750Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T10:44:18.061716586Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T10:44:18.138380281Z Gathering data for ns/default...
2026-07-28T10:44:18.448973032Z Gathering data for ns/openshift...
2026-07-28T10:44:18.895558427Z Gathering data for ns/kube-system...
2026-07-28T10:44:19.652619585Z Gathering data for ns/openshift-etcd...
2026-07-28T10:44:19.930546073Z Wrote inspect data to must-gather.
2026-07-28T10:44:19.930580919Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T10:44:19.930580919Z     namespaces "assisted-installer" not found
2026-07-28T10:44:20.409928904Z Waiting on subprocesses to finish execution.
2026-07-28T10:44:20.418878233Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-28T10:44:20.420073998Z INFO: Gathering HAProxy config files
2026-07-28T10:44:20.421231268Z Executing Istio gather script
2026-07-28T10:44:20.423273723Z INFO: Collecting host service logs for crio
2026-07-28T10:44:20.423624020Z INFO: Collecting host service logs for kubelet
2026-07-28T10:44:20.423954463Z INFO: Collecting host service logs for rpm-ostreed
2026-07-28T10:44:20.424482029Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-28T10:44:20.424829217Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-28T10:44:20.425160423Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-28T10:44:20.425649017Z INFO: Collecting host service logs for NetworkManager
2026-07-28T10:44:20.425986396Z INFO: Collecting host service logs for openvswitch
2026-07-28T10:44:20.426327287Z INFO: Collecting host service logs for ovs-configuration
2026-07-28T10:44:20.426677777Z INFO: Collecting host service logs for ovsdb-server
2026-07-28T10:44:20.427033128Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-28T10:44:20.427604448Z INFO: Waiting for worker host service log collection to complete ...
2026-07-28T10:44:20.434990245Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-28T10:44:20.463314343Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-28T10:44:20.644029410Z INFO: Waiting for node performance related collection to complete ...
2026-07-28T10:44:20.937020988Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-28T10:44:21.086705035Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-jd9hz
2026-07-28T10:44:21.086761921Z insights-runtime-extractor-gdrvq
2026-07-28T10:44:21.086781475Z insights-runtime-extractor-h9wkh
2026-07-28T10:44:21.086800707Z insights-runtime-extractor-q5dpw
2026-07-28T10:44:21.095994272Z No resources found in openshift-etcd namespace.
2026-07-28T10:44:21.456621648Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-28T10:44:21.539443426Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-28T10:44:21.624996447Z ERROR: No running kube-apiserver pods found
2026-07-28T10:44:21.693659802Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-28T10:44:21.693659802Z INFO: on-disk MachineConfig config collection complete.
2026-07-28T10:44:21.755807877Z Wrote inspect data to must-gather.
2026-07-28T10:44:21.906786567Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-28T10:44:21.999239557Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T10:44:22.043640544Z No resources found
2026-07-28T10:44:22.059989374Z No resources found
2026-07-28T10:44:22.130257790Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-28T10:44:22.130257790Z See 'oc rsync -h' for help and examples
2026-07-28T10:44:22.178270568Z INFO: Worker host service log collection to complete.
2026-07-28T10:44:22.187685338Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-28T10:44:22.234756203Z tar: Removing leading `/' from member names
2026-07-28T10:44:22.241225707Z INFO: HAProxy config collection complete.
2026-07-28T10:44:22.295351347Z error: the server doesn't have a resource type "performanceprofile"
2026-07-28T10:44:22.452909222Z INFO: "metallb-operator" not detected. Skipping.
2026-07-28T10:44:22.666544525Z INFO: 'previous-logs' folder not found on ip-10-0-130-249.ec2.internal, skipping...
2026-07-28T10:44:22.685060226Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-28T10:44:22.819685780Z Wrote inspect data to must-gather.
2026-07-28T10:44:22.839136647Z Wrote inspect data to must-gather.
2026-07-28T10:44:22.857045794Z Wrote inspect data to must-gather.
2026-07-28T10:44:22.890889817Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-28T10:44:22.994633071Z Wrote inspect data to must-gather.
2026-07-28T10:44:23.012201840Z Wrote inspect data to must-gather.
2026-07-28T10:44:23.076240270Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-28T10:44:23.136617490Z Wrote inspect data to must-gather.
2026-07-28T10:44:23.198941810Z INFO: 'previous-logs' folder not found on ip-10-0-132-110.ec2.internal, skipping...
2026-07-28T10:44:23.346590696Z error: the server doesn't have a resource type "machineconfigs"
2026-07-28T10:44:23.367123476Z Wrote inspect data to must-gather.
2026-07-28T10:44:23.426622149Z Gathering data for ns/openshift-monitoring...
2026-07-28T10:44:23.537343924Z INFO: 'previous-logs' folder not found on ip-10-0-135-126.ec2.internal, skipping...
2026-07-28T10:44:23.537483447Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-28T10:44:23.537543889Z INFO: Machine Config Daemon termination log collection complete.
2026-07-28T10:44:23.577978002Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-28T10:44:23.602205096Z INFO: Getting rules from prometheus-k8s-0
2026-07-28T10:44:23.678959444Z Wrote inspect data to must-gather.
2026-07-28T10:44:23.714904219Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-28T10:44:23.816933866Z Wrote inspect data to must-gather.
2026-07-28T10:44:23.956409527Z Wrote inspect data to must-gather.
2026-07-28T10:44:24.002575773Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-28T10:44:24.026734829Z INFO: Getting status/config from prometheus-k8s-0
2026-07-28T10:44:24.237752134Z Wrote inspect data to must-gather.
2026-07-28T10:44:24.292279744Z INFO: INTERCONNECT MODE
2026-07-28T10:44:24.292313259Z INFO: Gathering ovn-kubernetes DBs
2026-07-28T10:44:24.474568044Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-28T10:44:24.476689977Z INFO: Gathering OVN_Northbound from ovnkube-node-h9jtv...
2026-07-28T10:44:24.476985883Z INFO: Gathering OVN_Northbound from ovnkube-node-rrxnl...
2026-07-28T10:44:24.477309646Z INFO: Gathering OVN_Northbound from ovnkube-node-wkd29...
2026-07-28T10:44:24.477692853Z INFO: Gathering OVN_Southbound from ovnkube-node-h9jtv...
2026-07-28T10:44:24.477990072Z INFO: Gathering OVN_Southbound from ovnkube-node-rrxnl...
2026-07-28T10:44:24.478276704Z INFO: Gathering OVN_Southbound from ovnkube-node-wkd29...
2026-07-28T10:44:24.479845674Z Wrote inspect data to must-gather.
2026-07-28T10:44:24.786656629Z tar: Removing leading `/' from member names
2026-07-28T10:44:24.796373001Z tar: Removing leading `/' from member names
2026-07-28T10:44:24.816624872Z tar: Removing leading `/' from member names
2026-07-28T10:44:24.828635781Z tar: Removing leading `/' from member names
2026-07-28T10:44:24.864460308Z tar: Removing leading `/' from member names
2026-07-28T10:44:24.888341378Z tar: Removing leading `/' from member names
2026-07-28T10:44:24.908691167Z Wrote inspect data to must-gather.
2026-07-28T10:44:24.939769853Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-28T10:44:24.994960316Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T10:44:25.153000061Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-28T10:44:25.170108623Z Wrote inspect data to must-gather.
2026-07-28T10:44:25.224520492Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-28T10:44:25.404256546Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:25.449175731Z Wrote inspect data to must-gather.
2026-07-28T10:44:25.481320942Z INFO: Waiting for network log collection to complete ...
2026-07-28T10:44:25.481483337Z INFO: Waiting for ovnk database copies to complete ...
2026-07-28T10:44:25.481959036Z INFO: Copying ovnk databases complete.
2026-07-28T10:44:25.484146640Z 3.5M	must-gather/network_logs/ovnk_database_store
2026-07-28T10:44:25.489189704Z ovnk_database_store/
2026-07-28T10:44:25.489215392Z ovnk_database_store/ovnkube-node-rrxnl_sbdb
2026-07-28T10:44:25.517271824Z ovnk_database_store/ovnkube-node-h9jtv_sbdb
2026-07-28T10:44:25.518775895Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-28T10:44:25.538097752Z ovnk_database_store/ovnkube-node-rrxnl_nbdb
2026-07-28T10:44:25.547127541Z ovnk_database_store/ovnkube-node-wkd29_sbdb
2026-07-28T10:44:25.572136107Z ovnk_database_store/ovnkube-node-wkd29_nbdb
2026-07-28T10:44:25.583557008Z ovnk_database_store/ovnkube-node-h9jtv_nbdb
2026-07-28T10:44:25.594579591Z INFO: Network log collection complete.
2026-07-28T10:44:25.654173626Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:25.659158209Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-28T10:44:25.693830481Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T10:44:25.793436852Z Wrote inspect data to must-gather.
2026-07-28T10:44:25.877528207Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:25.893365166Z Gathering data for ns/openshift-network-console...
2026-07-28T10:44:25.941142739Z INFO: Getting status from alertmanager-main-0
2026-07-28T10:44:26.127193743Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:26.129861634Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-28T10:44:26.151512634Z Wrote inspect data to must-gather.
2026-07-28T10:44:26.345363872Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:26.356695517Z Gathering data for ns/openshift-console-operator...
2026-07-28T10:44:26.460295223Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-28T10:44:26.463043422Z INFO: Fallback to identify the container image from release info
2026-07-28T10:44:26.570558695Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:26.573285606Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-28T10:44:26.753259060Z Gathering data for ns/openshift-console...
2026-07-28T10:44:26.797718874Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:27.000668597Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:27.005065269Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-07-28T10:44:27.098502561Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-28T10:44:27.200706045Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:27.245247864Z daemonset.apps/perf-node-gather-daemonset created
2026-07-28T10:44:27.267062772Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-28T10:44:27.393347306Z Waiting for performance profile collector pods to become ready: 1
2026-07-28T10:44:27.426056608Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:27.428462249Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-07-28T10:44:27.599250078Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:27.727546496Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-28T10:44:27.836920404Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:27.838188895Z Gathering data for ns/openshift-dns-operator...
2026-07-28T10:44:27.839927585Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-28T10:44:28.018976235Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:28.234279739Z Wrote inspect data to /must-gather/istio.
2026-07-28T10:44:28.236936152Z Gathering data for ns/openshift-dns...
2026-07-28T10:44:28.540376940Z Waiting for performance profile collector pods to become ready: 2
2026-07-28T10:44:28.864556393Z Gathering data for ns/openshift-image-registry...
2026-07-28T10:44:28.935974618Z Done executing Istio gather script
2026-07-28T10:44:29.325926919Z Gathering data for ns/openshift-ingress-operator...
2026-07-28T10:44:29.678663456Z Waiting for performance profile collector pods to become ready: 3
2026-07-28T10:44:29.683549376Z Gathering data for ns/openshift-ingress...
2026-07-28T10:44:29.986208723Z Gathering data for ns/openshift-ingress-canary...
2026-07-28T10:44:30.340660935Z Gathering data for ns/openshift-insights...
2026-07-28T10:44:30.809722654Z Waiting for performance profile collector pods to become ready: 4
2026-07-28T10:44:31.933281516Z Waiting for performance profile collector pods to become ready: 5
2026-07-28T10:44:32.716590298Z Gathering data for ns/kserve...
2026-07-28T10:44:32.732619656Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-28T10:44:33.063911038Z Waiting for performance profile collector pods to become ready: 6
2026-07-28T10:44:34.182255439Z Waiting for performance profile collector pods to become ready: 7
2026-07-28T10:44:34.294069328Z Gathering data for ns/openshift-config...
2026-07-28T10:44:34.593886559Z Gathering data for ns/openshift-config-managed...
2026-07-28T10:44:35.046505298Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-28T10:44:35.303003217Z Gathering data for ns/openshift-kube-apiserver...
2026-07-28T10:44:35.313628738Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-28T10:44:35.583549573Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-28T10:44:35.603007012Z Collecting performance related data for node ip-10-0-132-110.ec2.internal
2026-07-28T10:44:35.613479326Z Collecting performance related data for node ip-10-0-130-249.ec2.internal
2026-07-28T10:44:35.623268127Z Collecting performance related data for node ip-10-0-135-126.ec2.internal
2026-07-28T10:44:35.908047046Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-28T10:44:36.270026221Z Gathering data for ns/openshift-kube-scheduler...
2026-07-28T10:44:36.662122635Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-28T10:44:37.051518021Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-28T10:44:37.445775842Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-28T10:44:37.733692304Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-28T10:44:37.737456649Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-28T10:44:38.312444593Z Gathering data for ns/openshift-multus...
2026-07-28T10:44:39.783635258Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-28T10:44:40.975937648Z Gathering data for ns/openshift-host-network...
2026-07-28T10:44:41.295653849Z Gathering data for ns/openshift-network-diagnostics...
2026-07-28T10:44:41.779957556Z Gathering data for ns/openshift-network-node-identity...
2026-07-28T10:44:42.176743146Z Gathering data for ns/openshift-network-operator...
2026-07-28T10:44:42.515227821Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-28T10:44:42.743929383Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-28T10:44:42.802880301Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-28T10:44:43.257887488Z Gathering data for ns/openshift-apiserver-operator...
2026-07-28T10:44:43.543244769Z Gathering data for ns/openshift-apiserver...
2026-07-28T10:44:43.801794109Z Collecting kubelet logs for node ip-10-0-132-110.ec2.internal
2026-07-28T10:44:43.804941628Z Collecting kubelet logs for node ip-10-0-130-249.ec2.internal
2026-07-28T10:44:43.807130885Z Collecting kubelet logs for node ip-10-0-135-126.ec2.internal
2026-07-28T10:44:44.032068273Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-28T10:44:44.189698583Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-pp4wj namespace
2026-07-28T10:44:44.195640470Z INFO: Node performance data collection complete.
2026-07-28T10:44:44.301120586Z Gathering data for ns/openshift-controller-manager...
2026-07-28T10:44:44.567063929Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-28T10:44:45.164516034Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-28T10:44:45.429293869Z Gathering data for ns/openshift-service-ca-operator...
2026-07-28T10:44:45.736338560Z Gathering data for ns/openshift-service-ca...
2026-07-28T10:44:46.058507719Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-28T10:44:46.897377041Z Wrote inspect data to must-gather.
2026-07-28T10:44:46.897436422Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T10:44:46.897436422Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-28T10:44:47.029738784Z error: the server doesn't have a resource type "clusters"
2026-07-28T10:44:47.178578198Z Caches written to disk