2026-09-19T03:13:48.161338041Z [disk usage checker] Started
2026-09-19T03:13:48.164610534Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-09-19T03:13:48.440017983Z Gathering data for ns/openshift-cluster-version...
2026-09-19T03:13:48.555616873Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T03:13:48.632634860Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:13:48.705845549Z Gathering data for ns/default...
2026-09-19T03:13:48.973379044Z Gathering data for ns/openshift...
2026-09-19T03:13:49.348113133Z Gathering data for ns/kube-system...
2026-09-19T03:13:50.015147109Z Gathering data for ns/openshift-etcd...
2026-09-19T03:13:50.310665138Z Wrote inspect data to must-gather.
2026-09-19T03:13:50.310697153Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T03:13:50.310697153Z     namespaces "assisted-installer" not found
2026-09-19T03:13:50.823318395Z Waiting on subprocesses to finish execution.
2026-09-19T03:13:50.830827616Z Executing Istio gather script
2026-09-19T03:13:50.836708880Z INFO: Gathering HAProxy config files
2026-09-19T03:13:50.840378164Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-19T03:13:50.842256265Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-19T03:13:50.851748063Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-19T03:13:50.859730818Z INFO: Collecting host service logs for crio
2026-09-19T03:13:50.860072092Z INFO: Collecting host service logs for kubelet
2026-09-19T03:13:50.860456938Z INFO: Collecting host service logs for rpm-ostreed
2026-09-19T03:13:50.860816914Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-19T03:13:50.861163547Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-19T03:13:50.861544177Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-19T03:13:50.861987106Z INFO: Collecting host service logs for NetworkManager
2026-09-19T03:13:50.862323200Z INFO: Collecting host service logs for openvswitch
2026-09-19T03:13:50.862677843Z INFO: Collecting host service logs for ovs-configuration
2026-09-19T03:13:50.862999392Z INFO: Collecting host service logs for ovsdb-server
2026-09-19T03:13:50.863305107Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-19T03:13:50.863744455Z INFO: Waiting for worker host service log collection to complete ...
2026-09-19T03:13:50.868667444Z INFO: Waiting for node performance related collection to complete ...
2026-09-19T03:13:51.090473778Z No resources found in openshift-etcd namespace.
2026-09-19T03:13:51.173022913Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-19T03:13:51.173114967Z INFO: on-disk MachineConfig config collection complete.
2026-09-19T03:13:51.249465024Z No resources found
2026-09-19T03:13:51.260326573Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-vg4fs
2026-09-19T03:13:51.260407472Z insights-runtime-extractor-6qwbv
2026-09-19T03:13:51.260447409Z insights-runtime-extractor-gc79m
2026-09-19T03:13:51.260475200Z insights-runtime-extractor-w4s84
2026-09-19T03:13:51.450255952Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-19T03:13:51.728633180Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-19T03:13:51.829122780Z error: the server doesn't have a resource type "performanceprofile"
2026-09-19T03:13:51.931233183Z ERROR: No running kube-apiserver pods found
2026-09-19T03:13:52.010612615Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-19T03:13:52.072661805Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-19T03:13:52.072661805Z See 'oc rsync -h' for help and examples
2026-09-19T03:13:52.175639125Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-19T03:13:52.213246867Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-19T03:13:52.244015689Z INFO: "metallb-operator" not detected. Skipping.
2026-09-19T03:13:52.261845587Z INFO: Worker host service log collection to complete.
2026-09-19T03:13:52.347963035Z No resources found
2026-09-19T03:13:52.653961403Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-19T03:13:52.784419405Z Wrote inspect data to must-gather.
2026-09-19T03:13:52.814127150Z Wrote inspect data to must-gather.
2026-09-19T03:13:52.856806380Z Wrote inspect data to must-gather.
2026-09-19T03:13:52.861594567Z Wrote inspect data to must-gather.
2026-09-19T03:13:52.935149851Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:13:53.060463957Z Gathering data for ns/openshift-monitoring...
2026-09-19T03:13:53.151009338Z tar: Removing leading `/' from member names
2026-09-19T03:13:53.158226560Z INFO: HAProxy config collection complete.
2026-09-19T03:13:53.187734635Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-09-19T03:13:53.258028946Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-19T03:13:53.329273482Z Wrote inspect data to must-gather.
2026-09-19T03:13:53.342973195Z error: the server doesn't have a resource type "machineconfigs"
2026-09-19T03:13:53.347612631Z Wrote inspect data to must-gather.
2026-09-19T03:13:53.368785142Z INFO: 'previous-logs' folder not found on ip-10-0-133-202.ec2.internal, skipping...
2026-09-19T03:13:53.404904214Z Wrote inspect data to must-gather.
2026-09-19T03:13:53.534626637Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-19T03:13:53.641174175Z Wrote inspect data to must-gather.
2026-09-19T03:13:53.685627130Z INFO: 'previous-logs' folder not found on ip-10-0-140-189.ec2.internal, skipping...
2026-09-19T03:13:53.772068196Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-19T03:13:53.779278986Z INFO: Getting rules from prometheus-k8s-0
2026-09-19T03:13:53.781118505Z Wrote inspect data to must-gather.
2026-09-19T03:13:53.937456428Z Wrote inspect data to must-gather.
2026-09-19T03:13:53.953578912Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-19T03:13:54.003585892Z INFO: 'previous-logs' folder not found on ip-10-0-142-14.ec2.internal, skipping...
2026-09-19T03:13:54.003623972Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-19T03:13:54.003649799Z INFO: Machine Config Daemon termination log collection complete.
2026-09-19T03:13:54.182417979Z Wrote inspect data to must-gather.
2026-09-19T03:13:54.185855670Z Wrote inspect data to must-gather.
2026-09-19T03:13:54.195106116Z INFO: Getting status/config from prometheus-k8s-0
2026-09-19T03:13:54.408489223Z Wrote inspect data to must-gather.
2026-09-19T03:13:54.494543039Z INFO: INTERCONNECT MODE
2026-09-19T03:13:54.494586139Z INFO: Gathering ovn-kubernetes DBs
2026-09-19T03:13:54.581737713Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-19T03:13:54.622353839Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-19T03:13:54.668197758Z Wrote inspect data to must-gather.
2026-09-19T03:13:54.668875627Z INFO: Gathering OVN_Northbound from ovnkube-node-4l7qd...
2026-09-19T03:13:54.669097697Z INFO: Gathering OVN_Northbound from ovnkube-node-jc9p9...
2026-09-19T03:13:54.669343140Z INFO: Gathering OVN_Northbound from ovnkube-node-jm7f2...
2026-09-19T03:13:54.669784674Z INFO: Gathering OVN_Southbound from ovnkube-node-4l7qd...
2026-09-19T03:13:54.670151484Z INFO: Gathering OVN_Southbound from ovnkube-node-jc9p9...
2026-09-19T03:13:54.670455463Z INFO: Gathering OVN_Southbound from ovnkube-node-jm7f2...
2026-09-19T03:13:54.671717055Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-19T03:13:54.969862338Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-19T03:13:54.991513598Z tar: Removing leading `/' from member names
2026-09-19T03:13:55.016896515Z tar: Removing leading `/' from member names
2026-09-19T03:13:55.017549039Z tar: Removing leading `/' from member names
2026-09-19T03:13:55.020953582Z tar: Removing leading `/' from member names
2026-09-19T03:13:55.057919834Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:55.070802359Z tar: Removing leading `/' from member names
2026-09-19T03:13:55.095134422Z Wrote inspect data to must-gather.
2026-09-19T03:13:55.098928505Z tar: Removing leading `/' from member names
2026-09-19T03:13:55.251432443Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-19T03:13:55.329770059Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:55.334307194Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-19T03:13:55.366064341Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-19T03:13:55.454637302Z Wrote inspect data to must-gather.
2026-09-19T03:13:55.538080782Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-19T03:13:55.583005977Z INFO: Waiting for network log collection to complete ...
2026-09-19T03:13:55.583119406Z INFO: Waiting for ovnk database copies to complete ...
2026-09-19T03:13:55.583626752Z INFO: Copying ovnk databases complete.
2026-09-19T03:13:55.588538851Z 3.0M	must-gather/network_logs/ovnk_database_store
2026-09-19T03:13:55.589370226Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:55.590221035Z Gathering data for ns/openshift-network-console...
2026-09-19T03:13:55.590918846Z ovnk_database_store/
2026-09-19T03:13:55.590935154Z ovnk_database_store/ovnkube-node-4l7qd_nbdb
2026-09-19T03:13:55.599502095Z ovnk_database_store/ovnkube-node-jc9p9_nbdb
2026-09-19T03:13:55.610118859Z ovnk_database_store/ovnkube-node-jm7f2_nbdb
2026-09-19T03:13:55.617417355Z ovnk_database_store/ovnkube-node-jc9p9_sbdb
2026-09-19T03:13:55.634778155Z ovnk_database_store/ovnkube-node-jm7f2_sbdb
2026-09-19T03:13:55.656079369Z ovnk_database_store/ovnkube-node-4l7qd_sbdb
2026-09-19T03:13:55.680410930Z INFO: Network log collection complete.
2026-09-19T03:13:55.783568743Z Wrote inspect data to must-gather.
2026-09-19T03:13:55.814522689Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:55.817860782Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-19T03:13:55.948965494Z INFO: Getting status from alertmanager-main-0
2026-09-19T03:13:56.031471683Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:56.096633412Z Wrote inspect data to must-gather.
2026-09-19T03:13:56.128850970Z Gathering data for ns/openshift-console-operator...
2026-09-19T03:13:56.249087601Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:56.252581444Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-19T03:13:56.357204034Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-19T03:13:56.359948056Z INFO: Fallback to identify the container image from release info
2026-09-19T03:13:56.461391475Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:56.512404123Z Gathering data for ns/openshift-console...
2026-09-19T03:13:56.661048293Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:56.664248675Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-19T03:13:56.809487374Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:57.008318188Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea
2026-09-19T03:13:57.016554041Z Wrote inspect data to /must-gather/istio.
2026-09-19T03:13:57.028709590Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-19T03:13:57.142214199Z daemonset.apps/perf-node-gather-daemonset created
2026-09-19T03:13:57.268345589Z Waiting for performance profile collector pods to become ready: 1
2026-09-19T03:13:57.556456882Z Gathering data for ns/openshift-dns-operator...
2026-09-19T03:13:57.713055339Z Done executing Istio gather script
2026-09-19T03:13:57.860014163Z Gathering data for ns/openshift-dns...
2026-09-19T03:13:58.194301150Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-09-19T03:13:58.392494554Z Waiting for performance profile collector pods to become ready: 2
2026-09-19T03:13:58.409020906Z Gathering data for ns/openshift-image-registry...
2026-09-19T03:13:58.803731982Z Gathering data for ns/openshift-ingress-operator...
2026-09-19T03:13:59.083109667Z Gathering data for ns/openshift-ingress...
2026-09-19T03:13:59.356245878Z Gathering data for ns/openshift-ingress-canary...
2026-09-19T03:13:59.513117934Z Waiting for performance profile collector pods to become ready: 3
2026-09-19T03:13:59.701303386Z Gathering data for ns/openshift-insights...
2026-09-19T03:14:00.618562511Z Waiting for performance profile collector pods to become ready: 4
2026-09-19T03:14:01.422882966Z Gathering data for ns/openshift-jobset-operator...
2026-09-19T03:14:01.736404484Z Waiting for performance profile collector pods to become ready: 5
2026-09-19T03:14:02.206286737Z Gathering data for ns/openshift-config...
2026-09-19T03:14:02.540831079Z Gathering data for ns/openshift-config-managed...
2026-09-19T03:14:02.857539590Z Waiting for performance profile collector pods to become ready: 6
2026-09-19T03:14:02.939798545Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-19T03:14:03.180506715Z Gathering data for ns/openshift-kube-apiserver...
2026-09-19T03:14:03.200489583Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-09-19T03:14:03.406888853Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-19T03:14:03.697392713Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-19T03:14:03.955193671Z Gathering data for ns/openshift-kube-scheduler...
2026-09-19T03:14:03.984447620Z Waiting for performance profile collector pods to become ready: 7
2026-09-19T03:14:04.221461297Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-19T03:14:04.459632621Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-19T03:14:04.731207943Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-19T03:14:05.040234332Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-19T03:14:05.116299941Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-19T03:14:05.367767015Z Collecting performance related data for node ip-10-0-142-14.ec2.internal
2026-09-19T03:14:05.374811235Z Collecting performance related data for node ip-10-0-140-189.ec2.internal
2026-09-19T03:14:05.383006771Z Collecting performance related data for node ip-10-0-133-202.ec2.internal
2026-09-19T03:14:05.561202171Z Gathering data for ns/openshift-multus...
2026-09-19T03:14:06.983541514Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-19T03:14:08.191666553Z Gathering data for ns/openshift-host-network...
2026-09-19T03:14:08.208145303Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-09-19T03:14:08.513224124Z Gathering data for ns/openshift-network-diagnostics...
2026-09-19T03:14:08.998094449Z Gathering data for ns/openshift-network-node-identity...
2026-09-19T03:14:09.394995242Z Gathering data for ns/openshift-network-operator...
2026-09-19T03:14:09.757127213Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-19T03:14:10.040234895Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-19T03:14:10.508561036Z Gathering data for ns/openshift-apiserver-operator...
2026-09-19T03:14:10.782053469Z Gathering data for ns/openshift-apiserver...
2026-09-19T03:14:11.145282056Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-19T03:14:11.374839546Z Gathering data for ns/openshift-controller-manager...
2026-09-19T03:14:11.628030689Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-19T03:14:12.281956937Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-19T03:14:12.629950646Z Gathering data for ns/openshift-service-ca-operator...
2026-09-19T03:14:12.901025685Z Gathering data for ns/openshift-service-ca...
2026-09-19T03:14:13.213821355Z [disk usage checker] Volume usage percentage: current = 28 ; allowed = 70
2026-09-19T03:14:13.216002396Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-19T03:14:13.791170169Z Collecting kubelet logs for node ip-10-0-142-14.ec2.internal
2026-09-19T03:14:13.792869240Z Collecting kubelet logs for node ip-10-0-140-189.ec2.internal
2026-09-19T03:14:13.794549736Z Collecting kubelet logs for node ip-10-0-133-202.ec2.internal
2026-09-19T03:14:14.105523869Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-wrwth namespace
2026-09-19T03:14:14.110101712Z INFO: Node performance data collection complete.
2026-09-19T03:14:14.110599438Z Wrote inspect data to must-gather.
2026-09-19T03:14:14.110660978Z error: inspection completed with the errors occurred while gathering data:
2026-09-19T03:14:14.110660978Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-19T03:14:14.234154404Z error: the server doesn't have a resource type "clusters"
2026-09-19T03:14:14.361900760Z Caches written to disk