2026-09-09T20:38:45.536969894Z [disk usage checker] Started
2026-09-09T20:38:45.540422942Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-09T20:38:45.742952494Z Gathering data for ns/openshift-cluster-version...
2026-09-09T20:38:45.873193352Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-09T20:38:45.950872404Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-09T20:38:46.029552780Z Gathering data for ns/default...
2026-09-09T20:38:46.341557576Z Gathering data for ns/openshift...
2026-09-09T20:38:46.765640448Z Gathering data for ns/kube-system...
2026-09-09T20:38:47.386769253Z Gathering data for ns/openshift-etcd...
2026-09-09T20:38:47.706492414Z Wrote inspect data to must-gather.
2026-09-09T20:38:47.706528860Z error: inspection completed with the errors occurred while gathering data:
2026-09-09T20:38:47.706528860Z     namespaces "assisted-installer" not found
2026-09-09T20:38:48.388449758Z Waiting on subprocesses to finish execution.
2026-09-09T20:38:48.394314301Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-09T20:38:48.400331424Z Executing Istio gather script
2026-09-09T20:38:48.403534853Z INFO: Collecting host service logs for crio
2026-09-09T20:38:48.403918626Z INFO: Collecting host service logs for kubelet
2026-09-09T20:38:48.404302760Z INFO: Collecting host service logs for rpm-ostreed
2026-09-09T20:38:48.404653731Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-09T20:38:48.404991244Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-09T20:38:48.405351784Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-09T20:38:48.405702638Z INFO: Collecting host service logs for NetworkManager
2026-09-09T20:38:48.406034361Z INFO: Collecting host service logs for openvswitch
2026-09-09T20:38:48.406451634Z INFO: Collecting host service logs for ovs-configuration
2026-09-09T20:38:48.406810221Z INFO: Collecting host service logs for ovsdb-server
2026-09-09T20:38:48.407173298Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-09T20:38:48.407583337Z INFO: Waiting for worker host service log collection to complete ...
2026-09-09T20:38:48.409688938Z INFO: Gathering HAProxy config files
2026-09-09T20:38:48.417943049Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-09T20:38:48.444265851Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-09T20:38:48.679501338Z INFO: Waiting for node performance related collection to complete ...
2026-09-09T20:38:48.745654041Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-09T20:38:49.382512337Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-09T20:38:49.382594499Z INFO: on-disk MachineConfig config collection complete.
2026-09-09T20:38:49.402273245Z No resources found in openshift-etcd namespace.
2026-09-09T20:38:49.459899899Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-09T20:38:49.595572344Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-nh9z8
2026-09-09T20:38:49.595638776Z insights-runtime-extractor-66t2s
2026-09-09T20:38:49.595659278Z insights-runtime-extractor-nw29n
2026-09-09T20:38:49.595680053Z insights-runtime-extractor-sq74v
2026-09-09T20:38:49.598130324Z INFO: "metallb-operator" not detected. Skipping.
2026-09-09T20:38:49.682921767Z No resources found
2026-09-09T20:38:49.709002829Z Wrote inspect data to must-gather.
2026-09-09T20:38:49.916050006Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-09T20:38:50.056278360Z INFO: Worker host service log collection to complete.
2026-09-09T20:38:50.074859884Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-09T20:38:50.107430192Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-09T20:38:50.154777554Z error: the server doesn't have a resource type "performanceprofile"
2026-09-09T20:38:50.303534620Z Wrote inspect data to must-gather.
2026-09-09T20:38:50.380540900Z No resources found
2026-09-09T20:38:50.616361030Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-09T20:38:50.650342845Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-09T20:38:50.717788503Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-09T20:38:50.721583151Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-09T20:38:50.721583151Z See 'oc rsync -h' for help and examples
2026-09-09T20:38:50.810282799Z Gathering data for ns/openshift-monitoring...
2026-09-09T20:38:50.836870653Z Wrote inspect data to must-gather.
2026-09-09T20:38:50.975740616Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-09T20:38:51.003188178Z Wrote inspect data to must-gather.
2026-09-09T20:38:51.040230969Z INFO: 'previous-logs' folder not found on ip-10-0-132-109.ec2.internal, skipping...
2026-09-09T20:38:51.064910643Z Wrote inspect data to must-gather.
2026-09-09T20:38:51.176043420Z ERROR: No running kube-apiserver pods found
2026-09-09T20:38:51.216226454Z Wrote inspect data to must-gather.
2026-09-09T20:38:51.250742640Z Wrote inspect data to must-gather.
2026-09-09T20:38:51.289526609Z INFO: Getting rules from prometheus-k8s-0
2026-09-09T20:38:51.421592432Z Wrote inspect data to must-gather.
2026-09-09T20:38:51.424719122Z tar: Removing leading `/' from member names
2026-09-09T20:38:51.429683773Z INFO: HAProxy config collection complete.
2026-09-09T20:38:51.468418699Z error: the server doesn't have a resource type "machineconfigs"
2026-09-09T20:38:51.501834901Z INFO: 'previous-logs' folder not found on ip-10-0-143-149.ec2.internal, skipping...
2026-09-09T20:38:51.699917015Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-09T20:38:51.746948708Z Wrote inspect data to must-gather.
2026-09-09T20:38:51.801296292Z INFO: Getting status/config from prometheus-k8s-0
2026-09-09T20:38:51.802574913Z INFO: 'previous-logs' folder not found on ip-10-0-143-188.ec2.internal, skipping...
2026-09-09T20:38:51.802605278Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-09T20:38:51.802635237Z INFO: Machine Config Daemon termination log collection complete.
2026-09-09T20:38:51.975926692Z Wrote inspect data to must-gather.
2026-09-09T20:38:52.047820328Z Wrote inspect data to must-gather.
2026-09-09T20:38:52.193742900Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-09T20:38:52.304557154Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-09T20:38:52.340464221Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-09T20:38:52.349864733Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-09T20:38:52.455108873Z INFO: INTERCONNECT MODE
2026-09-09T20:38:52.455163452Z INFO: Gathering ovn-kubernetes DBs
2026-09-09T20:38:52.493651872Z Wrote inspect data to must-gather.
2026-09-09T20:38:52.574247750Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-09T20:38:52.624318419Z INFO: Gathering OVN_Northbound from ovnkube-node-2c279...
2026-09-09T20:38:52.624591462Z INFO: Gathering OVN_Northbound from ovnkube-node-q8td4...
2026-09-09T20:38:52.624808660Z INFO: Gathering OVN_Northbound from ovnkube-node-rk4kx...
2026-09-09T20:38:52.625085897Z INFO: Gathering OVN_Southbound from ovnkube-node-2c279...
2026-09-09T20:38:52.625380909Z INFO: Gathering OVN_Southbound from ovnkube-node-q8td4...
2026-09-09T20:38:52.625631000Z INFO: Gathering OVN_Southbound from ovnkube-node-rk4kx...
2026-09-09T20:38:52.844874374Z Wrote inspect data to must-gather.
2026-09-09T20:38:52.940866632Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-09T20:38:52.946535400Z tar: Removing leading `/' from member names
2026-09-09T20:38:52.998854144Z tar: Removing leading `/' from member names
2026-09-09T20:38:53.033468849Z tar: Removing leading `/' from member names
2026-09-09T20:38:53.039224143Z tar: Removing leading `/' from member names
2026-09-09T20:38:53.064277056Z tar: Removing leading `/' from member names
2026-09-09T20:38:53.071570017Z tar: Removing leading `/' from member names
2026-09-09T20:38:53.159883173Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-09T20:38:53.251239447Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-09T20:38:53.301462864Z Wrote inspect data to must-gather.
2026-09-09T20:38:53.404711154Z Gathering data for ns/openshift-network-console...
2026-09-09T20:38:53.438705196Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-09T20:38:53.661370130Z INFO: Waiting for network log collection to complete ...
2026-09-09T20:38:53.661505103Z INFO: Waiting for ovnk database copies to complete ...
2026-09-09T20:38:53.661989821Z INFO: Copying ovnk databases complete.
2026-09-09T20:38:53.665683767Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-09-09T20:38:53.668671862Z ovnk_database_store/
2026-09-09T20:38:53.668671862Z ovnk_database_store/ovnkube-node-q8td4_nbdb
2026-09-09T20:38:53.680844937Z ovnk_database_store/ovnkube-node-2c279_sbdb
2026-09-09T20:38:53.691213328Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:53.697856984Z Wrote inspect data to must-gather.
2026-09-09T20:38:53.704539029Z ovnk_database_store/ovnkube-node-rk4kx_nbdb
2026-09-09T20:38:53.714409232Z ovnk_database_store/ovnkube-node-rk4kx_sbdb
2026-09-09T20:38:53.717454281Z INFO: Getting status from alertmanager-main-0
2026-09-09T20:38:53.739477839Z ovnk_database_store/ovnkube-node-2c279_nbdb
2026-09-09T20:38:53.753597337Z ovnk_database_store/ovnkube-node-q8td4_sbdb
2026-09-09T20:38:53.784759793Z INFO: Network log collection complete.
2026-09-09T20:38:53.940680090Z Gathering data for ns/openshift-console-operator...
2026-09-09T20:38:53.951275920Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:53.953737902Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-09T20:38:54.069957390Z Wrote inspect data to must-gather.
2026-09-09T20:38:54.175810774Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:54.341892851Z Gathering data for ns/openshift-console...
2026-09-09T20:38:54.388862516Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:54.390305606Z Wrote inspect data to must-gather.
2026-09-09T20:38:54.391547249Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-09T20:38:54.600339329Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:54.720333420Z Wrote inspect data to must-gather.
2026-09-09T20:38:54.819486204Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:54.822685827Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-09T20:38:54.882850702Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-09T20:38:55.034268288Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-09T20:38:55.036496115Z INFO: Fallback to identify the container image from release info
2026-09-09T20:38:55.060960172Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:55.272752046Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:55.275373308Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-09T20:38:55.422812572Z Gathering data for ns/openshift-dns-operator...
2026-09-09T20:38:55.475508275Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:55.663543889Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399
2026-09-09T20:38:55.673015362Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-09T20:38:55.711235111Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:55.718802919Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-09T20:38:55.777849629Z Gathering data for ns/openshift-dns...
2026-09-09T20:38:55.821981897Z daemonset.apps/perf-node-gather-daemonset created
2026-09-09T20:38:55.933536328Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:55.956773830Z Waiting for performance profile collector pods to become ready: 1
2026-09-09T20:38:56.129476862Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:56.132017933Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-09T20:38:56.334312617Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:56.440180571Z Gathering data for ns/openshift-image-registry...
2026-09-09T20:38:56.544427733Z Wrote inspect data to /must-gather/istio.
2026-09-09T20:38:56.923506178Z Gathering data for ns/openshift-ingress-operator...
2026-09-09T20:38:57.103129631Z Waiting for performance profile collector pods to become ready: 2
2026-09-09T20:38:57.280832701Z Gathering data for ns/openshift-ingress...
2026-09-09T20:38:57.294131674Z Done executing Istio gather script
2026-09-09T20:38:57.585981552Z Gathering data for ns/openshift-ingress-canary...
2026-09-09T20:38:57.946648638Z Gathering data for ns/openshift-insights...
2026-09-09T20:38:58.236227372Z Waiting for performance profile collector pods to become ready: 3
2026-09-09T20:38:59.360951116Z Waiting for performance profile collector pods to become ready: 4
2026-09-09T20:39:00.333149899Z Gathering data for ns/kserve...
2026-09-09T20:39:00.488695452Z Waiting for performance profile collector pods to become ready: 5
2026-09-09T20:39:00.677936068Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-09T20:39:01.623012330Z Waiting for performance profile collector pods to become ready: 6
2026-09-09T20:39:02.012420908Z Gathering data for ns/openshift-config...
2026-09-09T20:39:02.263392072Z Gathering data for ns/openshift-config-managed...
2026-09-09T20:39:02.670746625Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-09T20:39:02.766581863Z Waiting for performance profile collector pods to become ready: 7
2026-09-09T20:39:02.957637272Z Gathering data for ns/openshift-kube-apiserver...
2026-09-09T20:39:03.243027412Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-09T20:39:03.543117738Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-09T20:39:03.810688724Z Gathering data for ns/openshift-kube-scheduler...
2026-09-09T20:39:03.914305450Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-09T20:39:04.110503451Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-09T20:39:04.193365975Z Collecting performance related data for node ip-10-0-143-188.ec2.internal
2026-09-09T20:39:04.205692354Z Collecting performance related data for node ip-10-0-132-109.ec2.internal
2026-09-09T20:39:04.217107420Z Collecting performance related data for node ip-10-0-143-149.ec2.internal
2026-09-09T20:39:04.483654482Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-09T20:39:04.860016091Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-09T20:39:05.227767750Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-09T20:39:05.683052770Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-09T20:39:05.853201269Z Gathering data for ns/openshift-multus...
2026-09-09T20:39:07.251350062Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-09T20:39:08.416642487Z Gathering data for ns/openshift-host-network...
2026-09-09T20:39:08.763780053Z Gathering data for ns/openshift-network-diagnostics...
2026-09-09T20:39:09.283305390Z Gathering data for ns/openshift-network-node-identity...
2026-09-09T20:39:09.694184318Z Gathering data for ns/openshift-network-operator...
2026-09-09T20:39:10.054934956Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-09T20:39:10.309572765Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-09T20:39:10.689265971Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-09-09T20:39:10.782186057Z Gathering data for ns/openshift-apiserver-operator...
2026-09-09T20:39:11.082497904Z Gathering data for ns/openshift-apiserver...
2026-09-09T20:39:11.452615460Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-09T20:39:11.705900304Z Gathering data for ns/openshift-controller-manager...
2026-09-09T20:39:11.958326378Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-09T20:39:12.593896425Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-09T20:39:12.755251428Z Collecting kubelet logs for node ip-10-0-143-188.ec2.internal
2026-09-09T20:39:12.757482744Z Collecting kubelet logs for node ip-10-0-132-109.ec2.internal
2026-09-09T20:39:12.759306456Z Collecting kubelet logs for node ip-10-0-143-149.ec2.internal
2026-09-09T20:39:12.919507042Z Gathering data for ns/openshift-service-ca-operator...
2026-09-09T20:39:13.204192318Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-6t7hr namespace
2026-09-09T20:39:13.209594394Z INFO: Node performance data collection complete.
2026-09-09T20:39:13.260652862Z Gathering data for ns/openshift-service-ca...
2026-09-09T20:39:13.623304311Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-09T20:39:14.472675742Z Wrote inspect data to must-gather.
2026-09-09T20:39:14.472722579Z error: inspection completed with the errors occurred while gathering data:
2026-09-09T20:39:14.472722579Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-09T20:39:14.610514213Z error: the server doesn't have a resource type "clusters"
2026-09-09T20:39:14.785965318Z Caches written to disk