2026-10-31T08:20:50.111583235Z [disk usage checker] Started 2026-10-31T08:20:50.114870801Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-10-31T08:20:50.331505964Z Gathering data for ns/openshift-cluster-version... 2026-10-31T08:20:50.473833650Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-31T08:20:50.561871295Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-31T08:20:50.665568888Z Gathering data for ns/default... 2026-10-31T08:20:50.992310776Z Gathering data for ns/openshift... 2026-10-31T08:20:51.437459837Z Gathering data for ns/kube-system... 2026-10-31T08:20:52.268055771Z Gathering data for ns/openshift-etcd... 2026-10-31T08:20:52.631647317Z Wrote inspect data to must-gather. 2026-10-31T08:20:52.631681051Z error: inspection completed with the errors occurred while gathering data: 2026-10-31T08:20:52.631681051Z namespaces "assisted-installer" not found 2026-10-31T08:20:52.958945116Z Waiting on subprocesses to finish execution. 2026-10-31T08:20:52.974672609Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-10-31T08:20:52.975066603Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-10-31T08:20:52.975504641Z INFO: Gathering machine config daemon's old logs from all nodes 2026-10-31T08:20:52.977072884Z INFO: Collecting host service logs for crio 2026-10-31T08:20:52.977329948Z INFO: Collecting host service logs for kubelet 2026-10-31T08:20:52.977654363Z INFO: Collecting host service logs for rpm-ostreed 2026-10-31T08:20:52.978002026Z INFO: Collecting host service logs for ostree-finalize-staged 2026-10-31T08:20:52.978287269Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-10-31T08:20:52.978626268Z INFO: Collecting host service logs for machine-config-daemon-host 2026-10-31T08:20:52.978935436Z INFO: Collecting host service logs for NetworkManager 2026-10-31T08:20:52.979308302Z INFO: Collecting host service logs for openvswitch 2026-10-31T08:20:52.979688216Z INFO: Collecting host service logs for ovs-configuration 2026-10-31T08:20:52.980002568Z INFO: Collecting host service logs for ovsdb-server 2026-10-31T08:20:52.980297264Z INFO: Collecting host service logs for ovs-vswitchd 2026-10-31T08:20:52.980670649Z INFO: Waiting for worker host service log collection to complete ... 2026-10-31T08:20:52.981120949Z Executing Istio gather script 2026-10-31T08:20:52.988056598Z INFO: Gathering HAProxy config files 2026-10-31T08:20:53.094408340Z INFO: Waiting for node performance related collection to complete ... 2026-10-31T08:20:53.552755366Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-10-31T08:20:53.552802786Z INFO: on-disk MachineConfig config collection complete. 2026-10-31T08:20:53.896548189Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-10-31T08:20:53.993857839Z INFO: "metallb-operator" not detected. Skipping. 2026-10-31T08:20:53.998978979Z INFO: "nfd" not detected. Skipping. 2026-10-31T08:20:54.006343027Z INFO: Found 1 replicas - prometheus-k8s-0 2026-10-31T08:20:54.052745996Z No resources found in openshift-etcd namespace. 2026-10-31T08:20:54.235124859Z INFO: "sriov-network-operator" not detected. Skipping. 2026-10-31T08:20:54.235536097Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-hz58r 2026-10-31T08:20:54.235584569Z insights-runtime-extractor-4kmnm 2026-10-31T08:20:54.235616411Z insights-runtime-extractor-ktl5p 2026-10-31T08:20:54.235635702Z insights-runtime-extractor-mvmr5 2026-10-31T08:20:54.249855295Z error: the server doesn't have a resource type "performanceprofile" 2026-10-31T08:20:54.305851456Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-10-31T08:20:54.391919078Z No resources found 2026-10-31T08:20:54.442342439Z INFO: Worker host service log collection to complete. 2026-10-31T08:20:54.509930925Z Wrote inspect data to must-gather. 2026-10-31T08:20:54.592869145Z ERROR: No running kube-apiserver pods found 2026-10-31T08:20:54.629143668Z INFO: Waiting for HAProxy config collection to complete ... 2026-10-31T08:20:55.011875087Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-10-31T08:20:55.011875087Z See 'oc rsync -h' for help and examples 2026-10-31T08:20:55.139392790Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-31T08:20:55.154313734Z Wrote inspect data to must-gather. 2026-10-31T08:20:55.202510826Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-10-31T08:20:55.360565345Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-10-31T08:20:55.436391402Z Wrote inspect data to must-gather. 2026-10-31T08:20:55.468122232Z Wrote inspect data to must-gather. 2026-10-31T08:20:55.486164893Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-10-31T08:20:55.535921282Z No resources found 2026-10-31T08:20:55.611546469Z Wrote inspect data to must-gather. 2026-10-31T08:20:55.706999787Z Wrote inspect data to must-gather. 2026-10-31T08:20:55.717206127Z Wrote inspect data to must-gather. 2026-10-31T08:20:55.775495426Z tar: Removing leading `/' from member names 2026-10-31T08:20:55.792714357Z INFO: HAProxy config collection complete. 2026-10-31T08:20:55.832066414Z Gathering data for ns/openshift-cloud-credential-operator... 2026-10-31T08:20:55.896496688Z error: the server doesn't have a resource type "machineconfigs" 2026-10-31T08:20:55.932153399Z INFO: 'previous-logs' folder not found on ip-10-0-130-48.us-east-2.compute.internal, skipping... 2026-10-31T08:20:56.041629617Z Wrote inspect data to must-gather. 2026-10-31T08:20:56.054362868Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-31T08:20:56.135154390Z INFO: Getting rules from prometheus-k8s-0 2026-10-31T08:20:56.180670939Z error: the server doesn't have a resource type "machineconfigpools" 2026-10-31T08:20:56.269517883Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-31T08:20:56.322705037Z INFO: 'previous-logs' folder not found on ip-10-0-134-26.us-east-2.compute.internal, skipping... 2026-10-31T08:20:56.400225150Z Wrote inspect data to must-gather. 2026-10-31T08:20:56.639558112Z Wrote inspect data to must-gather. 2026-10-31T08:20:56.770751335Z INFO: Getting status/config from prometheus-k8s-0 2026-10-31T08:20:56.772105208Z INFO: 'previous-logs' folder not found on ip-10-0-136-240.us-east-2.compute.internal, skipping... 2026-10-31T08:20:56.772145823Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-10-31T08:20:56.772176468Z INFO: Machine Config Daemon termination log collection complete. 2026-10-31T08:20:56.790530700Z Wrote inspect data to must-gather. 2026-10-31T08:20:56.841483326Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-10-31T08:20:56.846028120Z Gathering data for ns/openshift-monitoring... 2026-10-31T08:20:56.915712989Z error: the server doesn't have a resource type "kubeletconfigs" 2026-10-31T08:20:57.145287236Z INFO: LEGACY MODE 2026-10-31T08:20:57.145313517Z INFO: Gathering ovn-kubernetes master data 2026-10-31T08:20:57.181719453Z Wrote inspect data to must-gather. 2026-10-31T08:20:57.251571190Z INFO: Getting status/flags from prometheus-k8s-0 2026-10-31T08:20:57.441331564Z Wrote inspect data to must-gather. 2026-10-31T08:20:57.506309593Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-10-31T08:20:57.705441833Z Wrote inspect data to must-gather. 2026-10-31T08:20:57.731390618Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-10-31T08:20:57.987793823Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-10-31T08:20:57.993466607Z Wrote inspect data to must-gather. 2026-10-31T08:20:58.006026660Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io 2026-10-31T08:20:58.035258677Z INFO: Waiting for network log collection to complete ... 2026-10-31T08:20:58.035302112Z INFO: Waiting for ovnk database copies to complete ... 2026-10-31T08:20:58.035774920Z INFO: Copying ovnk databases complete. 2026-10-31T08:20:58.039098256Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory 2026-10-31T08:20:58.039725336Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 780 is not a child of this shell 2026-10-31T08:20:58.039725336Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 781 is not a child of this shell 2026-10-31T08:20:58.039725336Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 782 is not a child of this shell 2026-10-31T08:20:58.039725336Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 783 is not a child of this shell 2026-10-31T08:20:58.039748428Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 784 is not a child of this shell 2026-10-31T08:20:58.039748428Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 785 is not a child of this shell 2026-10-31T08:20:58.039748428Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 786 is not a child of this shell 2026-10-31T08:20:58.039748428Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 788 is not a child of this shell 2026-10-31T08:20:58.039748428Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 792 is not a child of this shell 2026-10-31T08:20:58.039748428Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 793 is not a child of this shell 2026-10-31T08:20:58.039748428Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 794 is not a child of this shell 2026-10-31T08:20:58.039757345Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 795 is not a child of this shell 2026-10-31T08:20:58.039757345Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 796 is not a child of this shell 2026-10-31T08:20:58.041148635Z tar: ovnk_database_store: Cannot stat: No such file or directory 2026-10-31T08:20:58.045808039Z tar: Exiting with failure status due to previous errors 2026-10-31T08:20:58.046319387Z INFO: Network log collection complete. 2026-10-31T08:20:58.209880256Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:58.235651269Z Wrote inspect data to must-gather. 2026-10-31T08:20:58.402263179Z INFO: Getting status from alertmanager-main-0 2026-10-31T08:20:58.462328011Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:58.466142043Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-10-31T08:20:58.477297912Z Wrote inspect data to must-gather. 2026-10-31T08:20:58.675119074Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:58.708988650Z Wrote inspect data to must-gather. 2026-10-31T08:20:58.908735322Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:58.911383252Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-10-31T08:20:59.029882676Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod 2026-10-31T08:20:59.118253857Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:59.168456902Z Gathering data for ns/openshift-network-console... 2026-10-31T08:20:59.189578706Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c 2026-10-31T08:20:59.345583489Z daemonset.apps/perf-node-gather-daemonset created 2026-10-31T08:20:59.353798778Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:59.356602533Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-10-31T08:20:59.499165611Z Waiting for performance profile collector pods to become ready: 1 2026-10-31T08:20:59.600919160Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:59.651674979Z Gathering data for ns/openshift-console-operator... 2026-10-31T08:20:59.820586114Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:20:59.823259106Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-10-31T08:21:00.069336280Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:00.089947042Z Gathering data for ns/openshift-console... 2026-10-31T08:21:00.210705454Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-10-31T08:21:00.274059101Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:00.276621847Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-10-31T08:21:00.450751898Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:00.652574874Z Waiting for performance profile collector pods to become ready: 2 2026-10-31T08:21:00.671119214Z Gathering data for ns/openshift-cluster-storage-operator... 2026-10-31T08:21:00.685333516Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:00.688115993Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-10-31T08:21:00.869162342Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:01.081084891Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:01.084952846Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-10-31T08:21:01.232957462Z Gathering data for ns/openshift-dns-operator... 2026-10-31T08:21:01.275667601Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:01.488557745Z Wrote inspect data to /must-gather/istio. 2026-10-31T08:21:01.609127992Z Gathering data for ns/openshift-dns... 2026-10-31T08:21:01.782169368Z Waiting for performance profile collector pods to become ready: 3 2026-10-31T08:21:02.275041232Z Gathering data for ns/openshift-image-registry... 2026-10-31T08:21:02.359811301Z Done executing Istio gather script 2026-10-31T08:21:02.767714945Z Gathering data for ns/openshift-ingress-operator... 2026-10-31T08:21:02.912073455Z Waiting for performance profile collector pods to become ready: 4 2026-10-31T08:21:03.111317600Z Gathering data for ns/openshift-ingress... 2026-10-31T08:21:03.451893484Z Gathering data for ns/openshift-ingress-canary... 2026-10-31T08:21:03.846025112Z Gathering data for ns/openshift-insights... 2026-10-31T08:21:04.025197017Z Waiting for performance profile collector pods to become ready: 5 2026-10-31T08:21:05.152015785Z Waiting for performance profile collector pods to become ready: 6 2026-10-31T08:21:05.215729551Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-10-31T08:21:06.288573347Z Waiting for performance profile collector pods to become ready: 7 2026-10-31T08:21:06.487156398Z Gathering data for ns/kserve... 2026-10-31T08:21:07.440197126Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-10-31T08:21:07.699725246Z Collecting performance related data for node ip-10-0-136-240.us-east-2.compute.internal 2026-10-31T08:21:07.706962092Z Collecting performance related data for node ip-10-0-130-48.us-east-2.compute.internal 2026-10-31T08:21:07.714950655Z Collecting performance related data for node ip-10-0-134-26.us-east-2.compute.internal 2026-10-31T08:21:08.304256619Z Gathering data for ns/openshift-config... 2026-10-31T08:21:08.690655418Z Gathering data for ns/openshift-config-managed... 2026-10-31T08:21:09.233652622Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-10-31T08:21:09.501029325Z Gathering data for ns/openshift-kube-apiserver... 2026-10-31T08:21:09.776893863Z Gathering data for ns/openshift-kube-controller-manager... 2026-10-31T08:21:10.140781443Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-10-31T08:21:10.223225672Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-10-31T08:21:10.533906012Z Gathering data for ns/openshift-kube-scheduler... 2026-10-31T08:21:10.822156998Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-10-31T08:21:11.117535806Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-10-31T08:21:11.435034126Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-10-31T08:21:11.743210964Z Gathering data for ns/openshift-user-workload-monitoring... 2026-10-31T08:21:12.234019592Z Gathering data for ns/openshift-multus... 2026-10-31T08:21:13.670164507Z Gathering data for ns/openshift-ovn-kubernetes... 2026-10-31T08:21:14.798552968Z Collecting kubelet logs for node ip-10-0-136-240.us-east-2.compute.internal 2026-10-31T08:21:14.800926345Z Collecting kubelet logs for node ip-10-0-130-48.us-east-2.compute.internal 2026-10-31T08:21:14.803151016Z Collecting kubelet logs for node ip-10-0-134-26.us-east-2.compute.internal 2026-10-31T08:21:15.053257668Z Gathering data for ns/openshift-host-network... 2026-10-31T08:21:15.229831928Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-10-31T08:21:15.264196005Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-xnprb namespace 2026-10-31T08:21:15.270809088Z INFO: Node performance data collection complete. 2026-10-31T08:21:15.415056035Z Gathering data for ns/openshift-network-diagnostics... 2026-10-31T08:21:15.988164734Z Gathering data for ns/openshift-network-node-identity... 2026-10-31T08:21:16.425578567Z Gathering data for ns/openshift-network-operator... 2026-10-31T08:21:16.800383957Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-10-31T08:21:17.072141045Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-10-31T08:21:17.533203914Z Gathering data for ns/openshift-apiserver-operator... 2026-10-31T08:21:17.818913769Z Gathering data for ns/openshift-apiserver... 2026-10-31T08:21:18.201421691Z Gathering data for ns/openshift-controller-manager-operator... 2026-10-31T08:21:18.474871727Z Gathering data for ns/openshift-controller-manager... 2026-10-31T08:21:18.742339390Z Gathering data for ns/openshift-cluster-samples-operator... 2026-10-31T08:21:19.397031611Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-10-31T08:21:19.717845857Z Gathering data for ns/openshift-service-ca-operator... 2026-10-31T08:21:20.018433015Z Gathering data for ns/openshift-service-ca... 2026-10-31T08:21:20.234912634Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-10-31T08:21:20.366008023Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-10-31T08:21:21.260730415Z Wrote inspect data to must-gather. 2026-10-31T08:21:21.260777127Z error: inspection completed with the errors occurred while gathering data: 2026-10-31T08:21:21.260777127Z skipping gathering secrets/support due to error: secrets "support" not found 2026-10-31T08:21:21.389489456Z error: the server doesn't have a resource type "clusters" 2026-10-31T08:21:21.535902738Z Caches written to disk