2026-07-28T11:54:53.592495140Z [disk usage checker] Started
2026-07-28T11:54:53.596052233Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-28T11:54:53.797725023Z Gathering data for ns/openshift-cluster-version...
2026-07-28T11:54:53.940655017Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T11:54:54.041088310Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T11:54:54.134741283Z Gathering data for ns/default...
2026-07-28T11:54:54.452086617Z Gathering data for ns/openshift...
2026-07-28T11:54:54.902142718Z Gathering data for ns/kube-system...
2026-07-28T11:54:55.674276616Z Gathering data for ns/openshift-etcd...
2026-07-28T11:54:56.008204081Z Wrote inspect data to must-gather.
2026-07-28T11:54:56.008240163Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T11:54:56.008240163Z     namespaces "assisted-installer" not found
2026-07-28T11:54:56.287589638Z Waiting on subprocesses to finish execution.
2026-07-28T11:54:56.293550647Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-28T11:54:56.297525613Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-28T11:54:56.297992114Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-28T11:54:56.305223639Z INFO: Gathering HAProxy config files
2026-07-28T11:54:56.305478569Z Executing Istio gather script
2026-07-28T11:54:56.338311730Z INFO: Collecting host service logs for crio
2026-07-28T11:54:56.339078731Z INFO: Collecting host service logs for kubelet
2026-07-28T11:54:56.339453382Z INFO: Collecting host service logs for rpm-ostreed
2026-07-28T11:54:56.339813405Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-28T11:54:56.340172154Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-28T11:54:56.340513359Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-28T11:54:56.340836648Z INFO: Collecting host service logs for NetworkManager
2026-07-28T11:54:56.341213939Z INFO: Collecting host service logs for openvswitch
2026-07-28T11:54:56.341590592Z INFO: Collecting host service logs for ovs-configuration
2026-07-28T11:54:56.342942186Z INFO: Collecting host service logs for ovsdb-server
2026-07-28T11:54:56.343292669Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-28T11:54:56.343680472Z INFO: Waiting for worker host service log collection to complete ...
2026-07-28T11:54:56.424012264Z INFO: Waiting for node performance related collection to complete ...
2026-07-28T11:54:56.729901603Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-hhfr2
2026-07-28T11:54:56.729965080Z insights-runtime-extractor-8pcfn
2026-07-28T11:54:56.730000316Z insights-runtime-extractor-mdvdv
2026-07-28T11:54:56.730022492Z insights-runtime-extractor-xpndg
2026-07-28T11:54:56.910600833Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-28T11:54:57.088922255Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-28T11:54:57.145026214Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-28T11:54:57.185411759Z INFO: "metallb-operator" not detected. Skipping.
2026-07-28T11:54:57.231998925Z Wrote inspect data to must-gather.
2026-07-28T11:54:57.275585812Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-28T11:54:57.275628065Z INFO: on-disk MachineConfig config collection complete.
2026-07-28T11:54:57.277998981Z No resources found in openshift-etcd namespace.
2026-07-28T11:54:57.300249772Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-28T11:54:57.300249772Z See 'oc rsync -h' for help and examples
2026-07-28T11:54:57.301156560Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-28T11:54:57.836763334Z No resources found
2026-07-28T11:54:57.952578668Z No resources found
2026-07-28T11:54:58.032358000Z Wrote inspect data to must-gather.
2026-07-28T11:54:58.126332433Z Wrote inspect data to must-gather.
2026-07-28T11:54:58.136324626Z ERROR: No running kube-apiserver pods found
2026-07-28T11:54:58.226490036Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-28T11:54:58.303356588Z error: the server doesn't have a resource type "performanceprofile"
2026-07-28T11:54:58.343919868Z INFO: Worker host service log collection to complete.
2026-07-28T11:54:58.387111828Z Wrote inspect data to must-gather.
2026-07-28T11:54:58.417103831Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-28T11:54:58.484849440Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-28T11:54:58.510271267Z Wrote inspect data to must-gather.
2026-07-28T11:54:58.555632181Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T11:54:58.580059584Z Wrote inspect data to must-gather.
2026-07-28T11:54:58.616320052Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-28T11:54:58.693292591Z INFO: 'previous-logs' folder not found on ip-10-0-128-89.ec2.internal, skipping...
2026-07-28T11:54:58.845717571Z Wrote inspect data to must-gather.
2026-07-28T11:54:58.855299188Z tar: Removing leading `/' from member names
2026-07-28T11:54:58.859377392Z INFO: HAProxy config collection complete.
2026-07-28T11:54:58.875559682Z Wrote inspect data to must-gather.
2026-07-28T11:54:59.072820426Z INFO: Getting rules from prometheus-k8s-0
2026-07-28T11:54:59.091240586Z INFO: 'previous-logs' folder not found on ip-10-0-132-238.ec2.internal, skipping...
2026-07-28T11:54:59.099878939Z error: the server doesn't have a resource type "machineconfigs"
2026-07-28T11:54:59.210206251Z Wrote inspect data to must-gather.
2026-07-28T11:54:59.216143606Z Gathering data for ns/openshift-monitoring...
2026-07-28T11:54:59.304913285Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-28T11:54:59.311924433Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-28T11:54:59.456287321Z INFO: 'previous-logs' folder not found on ip-10-0-139-249.ec2.internal, skipping...
2026-07-28T11:54:59.456321406Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-28T11:54:59.456383415Z INFO: Machine Config Daemon termination log collection complete.
2026-07-28T11:54:59.479737007Z Wrote inspect data to must-gather.
2026-07-28T11:54:59.525046391Z INFO: Getting status/config from prometheus-k8s-0
2026-07-28T11:54:59.559548267Z Wrote inspect data to must-gather.
2026-07-28T11:54:59.734802291Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-28T11:54:59.784829156Z INFO: INTERCONNECT MODE
2026-07-28T11:54:59.784866857Z INFO: Gathering ovn-kubernetes DBs
2026-07-28T11:54:59.945903882Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-28T11:54:59.947766242Z Wrote inspect data to must-gather.
2026-07-28T11:54:59.958498370Z INFO: Gathering OVN_Northbound from ovnkube-node-8vjzj...
2026-07-28T11:54:59.958738863Z INFO: Gathering OVN_Northbound from ovnkube-node-m9lsv...
2026-07-28T11:54:59.959088853Z INFO: Gathering OVN_Northbound from ovnkube-node-rwfc4...
2026-07-28T11:54:59.959305075Z INFO: Gathering OVN_Southbound from ovnkube-node-8vjzj...
2026-07-28T11:54:59.959616435Z INFO: Gathering OVN_Southbound from ovnkube-node-m9lsv...
2026-07-28T11:54:59.959909038Z INFO: Gathering OVN_Southbound from ovnkube-node-rwfc4...
2026-07-28T11:55:00.286033967Z tar: Removing leading `/' from member names
2026-07-28T11:55:00.296025276Z tar: Removing leading `/' from member names
2026-07-28T11:55:00.306762604Z Wrote inspect data to must-gather.
2026-07-28T11:55:00.317338550Z tar: Removing leading `/' from member names
2026-07-28T11:55:00.328274481Z tar: Removing leading `/' from member names
2026-07-28T11:55:00.338462062Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-28T11:55:00.346937607Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-28T11:55:00.374905061Z tar: Removing leading `/' from member names
2026-07-28T11:55:00.391678749Z tar: Removing leading `/' from member names
2026-07-28T11:55:00.541297375Z Wrote inspect data to must-gather.
2026-07-28T11:55:00.549016509Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:00.593545392Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-28T11:55:00.771346219Z Wrote inspect data to must-gather.
2026-07-28T11:55:00.778210300Z INFO: Waiting for network log collection to complete ...
2026-07-28T11:55:00.778240374Z INFO: Waiting for ovnk database copies to complete ...
2026-07-28T11:55:00.778653788Z INFO: Copying ovnk databases complete.
2026-07-28T11:55:00.780857109Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-07-28T11:55:00.784089011Z ovnk_database_store/
2026-07-28T11:55:00.784140882Z ovnk_database_store/ovnkube-node-8vjzj_sbdb
2026-07-28T11:55:00.787384093Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:00.790931319Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-28T11:55:00.807895756Z ovnk_database_store/ovnkube-node-rwfc4_nbdb
2026-07-28T11:55:00.815651251Z ovnk_database_store/ovnkube-node-8vjzj_nbdb
2026-07-28T11:55:00.824210025Z ovnk_database_store/ovnkube-node-rwfc4_sbdb
2026-07-28T11:55:00.845279464Z ovnk_database_store/ovnkube-node-m9lsv_nbdb
2026-07-28T11:55:00.853879772Z ovnk_database_store/ovnkube-node-m9lsv_sbdb
2026-07-28T11:55:00.858182047Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-28T11:55:00.876271121Z INFO: Network log collection complete.
2026-07-28T11:55:00.980216542Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-28T11:55:01.006912816Z Wrote inspect data to must-gather.
2026-07-28T11:55:01.012307647Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:01.236986915Z Wrote inspect data to must-gather.
2026-07-28T11:55:01.238110784Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:01.240798782Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-28T11:55:01.259156741Z INFO: Getting status from alertmanager-main-0
2026-07-28T11:55:01.442487332Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:01.461625579Z Wrote inspect data to must-gather.
2026-07-28T11:55:01.653484480Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-28T11:55:01.656485010Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:01.660522399Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-28T11:55:01.733546472Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-28T11:55:01.736652452Z INFO: Fallback to identify the container image from release info
2026-07-28T11:55:01.843835659Z Gathering data for ns/openshift-network-console...
2026-07-28T11:55:01.897225544Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:02.098452943Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:02.103146544Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-28T11:55:02.277742613Z Gathering data for ns/openshift-console-operator...
2026-07-28T11:55:02.281172788Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:02.475302394Z Wrote inspect data to /must-gather/istio.
2026-07-28T11:55:02.652751425Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-28T11:55:02.689252889Z Gathering data for ns/openshift-console...
2026-07-28T11:55:02.791450064Z daemonset.apps/perf-node-gather-daemonset created
2026-07-28T11:55:02.926081993Z Waiting for performance profile collector pods to become ready: 1
2026-07-28T11:55:03.221248078Z Done executing Istio gather script
2026-07-28T11:55:03.240546853Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-28T11:55:03.632700345Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-28T11:55:03.719844272Z Gathering data for ns/openshift-dns-operator...
2026-07-28T11:55:03.996401090Z Gathering data for ns/openshift-dns...
2026-07-28T11:55:04.045121179Z Waiting for performance profile collector pods to become ready: 2
2026-07-28T11:55:04.587872620Z Gathering data for ns/openshift-image-registry...
2026-07-28T11:55:05.043995318Z Gathering data for ns/openshift-ingress-operator...
2026-07-28T11:55:05.171027649Z Waiting for performance profile collector pods to become ready: 3
2026-07-28T11:55:05.377215587Z Gathering data for ns/openshift-ingress...
2026-07-28T11:55:05.720214288Z Gathering data for ns/openshift-ingress-canary...
2026-07-28T11:55:06.100431411Z Gathering data for ns/openshift-insights...
2026-07-28T11:55:06.276037163Z Waiting for performance profile collector pods to become ready: 4
2026-07-28T11:55:07.394487538Z Waiting for performance profile collector pods to become ready: 5
2026-07-28T11:55:07.932081060Z Gathering data for ns/openshift-jobset-operator...
2026-07-28T11:55:08.511207829Z Waiting for performance profile collector pods to become ready: 6
2026-07-28T11:55:08.637575317Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-28T11:55:08.750115793Z Gathering data for ns/openshift-config...
2026-07-28T11:55:09.076259912Z Gathering data for ns/openshift-config-managed...
2026-07-28T11:55:09.524671121Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-28T11:55:09.641788844Z Waiting for performance profile collector pods to become ready: 7
2026-07-28T11:55:09.837715447Z Gathering data for ns/openshift-kube-apiserver...
2026-07-28T11:55:10.125844881Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-28T11:55:10.423147823Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-28T11:55:10.775699989Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-28T11:55:10.797858226Z Gathering data for ns/openshift-kube-scheduler...
2026-07-28T11:55:11.033574379Z Collecting performance related data for node ip-10-0-139-249.ec2.internal
2026-07-28T11:55:11.043473923Z Collecting performance related data for node ip-10-0-132-238.ec2.internal
2026-07-28T11:55:11.054589998Z Collecting performance related data for node ip-10-0-128-89.ec2.internal
2026-07-28T11:55:11.197543174Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-28T11:55:11.581964600Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-28T11:55:11.980513351Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-28T11:55:12.392991630Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-28T11:55:12.941870055Z Gathering data for ns/openshift-multus...
2026-07-28T11:55:13.642328405Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-28T11:55:14.565566945Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-28T11:55:15.882659840Z Gathering data for ns/openshift-host-network...
2026-07-28T11:55:16.231462118Z Gathering data for ns/openshift-network-diagnostics...
2026-07-28T11:55:16.869575134Z Gathering data for ns/openshift-network-node-identity...
2026-07-28T11:55:17.418848332Z Gathering data for ns/openshift-network-operator...
2026-07-28T11:55:17.860623787Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-28T11:55:18.199272854Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-28T11:55:18.647579529Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-28T11:55:18.776333756Z Gathering data for ns/openshift-apiserver-operator...
2026-07-28T11:55:19.107663158Z Gathering data for ns/openshift-apiserver...
2026-07-28T11:55:19.503729140Z Collecting kubelet logs for node ip-10-0-139-249.ec2.internal
2026-07-28T11:55:19.505757299Z Collecting kubelet logs for node ip-10-0-132-238.ec2.internal
2026-07-28T11:55:19.508146785Z Collecting kubelet logs for node ip-10-0-128-89.ec2.internal
2026-07-28T11:55:19.538757575Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-28T11:55:19.890720831Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-qn8hn namespace
2026-07-28T11:55:19.903997181Z INFO: Node performance data collection complete.
2026-07-28T11:55:19.945413608Z Gathering data for ns/openshift-controller-manager...
2026-07-28T11:55:20.311260280Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-28T11:55:20.965688022Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-28T11:55:21.269040023Z Gathering data for ns/openshift-service-ca-operator...
2026-07-28T11:55:21.588108606Z Gathering data for ns/openshift-service-ca...
2026-07-28T11:55:21.959770115Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-28T11:55:22.968366446Z Wrote inspect data to must-gather.
2026-07-28T11:55:22.968418863Z error: inspection completed with the errors occurred while gathering data:
2026-07-28T11:55:22.968418863Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-28T11:55:23.097813786Z error: the server doesn't have a resource type "clusters"
2026-07-28T11:55:23.234724153Z Caches written to disk