2026-10-30T19:53:45.568378985Z [disk usage checker] Started 2026-10-30T19:53:45.571643040Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:53:46.056423187Z Gathering data for ns/openshift-cluster-version... 2026-10-30T19:53:46.384449968Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-30T19:53:46.585751936Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-30T19:53:46.786673125Z Gathering data for ns/default... 2026-10-30T19:53:47.530551706Z Gathering data for ns/openshift... 2026-10-30T19:53:48.349443817Z Gathering data for ns/kube-system... 2026-10-30T19:53:49.353380153Z Waiting on subprocesses to finish execution. 2026-10-30T19:53:49.359713291Z INFO: Gathering machine config daemon's old logs from all nodes 2026-10-30T19:53:49.360741120Z Executing Istio gather script 2026-10-30T19:53:49.363880509Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-10-30T19:53:49.369641241Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-10-30T19:53:49.370083258Z INFO: Gathering HAProxy config files 2026-10-30T19:53:49.393303134Z INFO: Collecting host service logs for crio 2026-10-30T19:53:49.393641839Z INFO: Collecting host service logs for kubelet 2026-10-30T19:53:49.393943736Z INFO: Collecting host service logs for rpm-ostreed 2026-10-30T19:53:49.394240435Z INFO: Collecting host service logs for ostree-finalize-staged 2026-10-30T19:53:49.394562267Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-10-30T19:53:49.394859307Z INFO: Collecting host service logs for machine-config-daemon-host 2026-10-30T19:53:49.395173486Z INFO: Collecting host service logs for NetworkManager 2026-10-30T19:53:49.395499460Z INFO: Collecting host service logs for openvswitch 2026-10-30T19:53:49.395817868Z INFO: Collecting host service logs for ovs-configuration 2026-10-30T19:53:49.396140917Z INFO: Collecting host service logs for ovsdb-server 2026-10-30T19:53:49.396477553Z INFO: Collecting host service logs for ovs-vswitchd 2026-10-30T19:53:49.396846328Z INFO: Waiting for worker host service log collection to complete ... 2026-10-30T19:53:49.575610402Z INFO: Waiting for node performance related collection to complete ... 2026-10-30T19:53:50.081277913Z No resources found in openshift-etcd namespace. 2026-10-30T19:53:50.167533883Z Wrote inspect data to must-gather. 2026-10-30T19:53:50.324083359Z INFO: "metallb-operator" not detected. Skipping. 2026-10-30T19:53:50.442545202Z No resources found 2026-10-30T19:53:50.708383142Z INFO: "sriov-network-operator" not detected. Skipping. 2026-10-30T19:53:50.750761781Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:53:50.776031339Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-10-30T19:53:50.776150293Z INFO: on-disk MachineConfig config collection complete. 2026-10-30T19:53:50.954673892Z error: the server doesn't have a resource type "performanceprofile" 2026-10-30T19:53:50.985626250Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-10-30T19:53:51.051507948Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-rdqbn 2026-10-30T19:53:51.051507948Z insights-runtime-extractor-9gqmm 2026-10-30T19:53:51.051507948Z insights-runtime-extractor-p9s6d 2026-10-30T19:53:51.051507948Z insights-runtime-extractor-qdjrf 2026-10-30T19:53:51.128831255Z INFO: "nfd" not detected. Skipping. 2026-10-30T19:53:51.253930184Z INFO: Found 1 replicas - prometheus-k8s-0 2026-10-30T19:53:51.295478773Z ERROR: No running kube-apiserver pods found 2026-10-30T19:53:51.313290429Z INFO: Worker host service log collection to complete. 2026-10-30T19:53:51.336004594Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-10-30T19:53:51.709763158Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-30T19:53:51.720457208Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-10-30T19:53:51.720457208Z See 'oc rsync -h' for help and examples 2026-10-30T19:53:51.842494061Z INFO: Waiting for HAProxy config collection to complete ... 2026-10-30T19:53:52.210397204Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-10-30T19:53:52.233576664Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-10-30T19:53:52.264423046Z No resources found 2026-10-30T19:53:52.297799806Z Wrote inspect data to must-gather. 2026-10-30T19:53:52.411085221Z Wrote inspect data to must-gather. 2026-10-30T19:53:52.413058051Z Wrote inspect data to must-gather. 2026-10-30T19:53:52.430169604Z Wrote inspect data to must-gather. 2026-10-30T19:53:52.449782611Z Gathering data for ns/openshift-etcd... 2026-10-30T19:53:52.561932089Z Wrote inspect data to must-gather. 2026-10-30T19:53:52.630556144Z tar: Removing leading `/' from member names 2026-10-30T19:53:52.645383219Z INFO: HAProxy config collection complete. 2026-10-30T19:53:52.660382358Z Gathering data for ns/openshift-cloud-credential-operator... 2026-10-30T19:53:52.701097302Z INFO: 'previous-logs' folder not found on ip-10-0-131-175.ec2.internal, skipping... 2026-10-30T19:53:52.964290414Z error: the server doesn't have a resource type "machineconfigs" 2026-10-30T19:53:52.993128586Z Wrote inspect data to must-gather. 2026-10-30T19:53:53.050702688Z Wrote inspect data to must-gather. 2026-10-30T19:53:53.116720362Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-30T19:53:53.157726893Z INFO: Getting rules from prometheus-k8s-0 2026-10-30T19:53:53.209492315Z INFO: 'previous-logs' folder not found on ip-10-0-139-153.ec2.internal, skipping... 2026-10-30T19:53:53.265267819Z error: the server doesn't have a resource type "machineconfigpools" 2026-10-30T19:53:53.308431932Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-10-30T19:53:53.435553656Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-30T19:53:53.447274643Z Wrote inspect data to must-gather. 2026-10-30T19:53:53.567459204Z Wrote inspect data to must-gather. 2026-10-30T19:53:53.567498196Z error: inspection completed with the errors occurred while gathering data: 2026-10-30T19:53:53.567498196Z namespaces "assisted-installer" not found 2026-10-30T19:53:53.586781783Z INFO: 'previous-logs' folder not found on ip-10-0-143-51.ec2.internal, skipping... 2026-10-30T19:53:53.586896081Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-10-30T19:53:53.586954178Z INFO: Machine Config Daemon termination log collection complete. 2026-10-30T19:53:53.634676572Z Wrote inspect data to must-gather. 2026-10-30T19:53:53.878302763Z Wrote inspect data to must-gather. 2026-10-30T19:53:53.884787665Z error: the server doesn't have a resource type "kubeletconfigs" 2026-10-30T19:53:53.911504485Z INFO: Getting status/config from prometheus-k8s-0 2026-10-30T19:53:54.183677876Z Gathering data for ns/openshift-monitoring... 2026-10-30T19:53:54.201579246Z Wrote inspect data to must-gather. 2026-10-30T19:53:54.316253972Z INFO: LEGACY MODE 2026-10-30T19:53:54.316293401Z INFO: Gathering ovn-kubernetes master data 2026-10-30T19:53:54.467834064Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io 2026-10-30T19:53:54.515381856Z Wrote inspect data to must-gather. 2026-10-30T19:53:54.590732064Z INFO: Getting status/flags from prometheus-k8s-0 2026-10-30T19:53:54.754407469Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:54.805729439Z Wrote inspect data to must-gather. 2026-10-30T19:53:54.960034454Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-10-30T19:53:55.105935067Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:55.110547907Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-10-30T19:53:55.152600482Z Wrote inspect data to must-gather. 2026-10-30T19:53:55.350061784Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:55.357909268Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-10-30T19:53:55.420760905Z INFO: Waiting for network log collection to complete ... 2026-10-30T19:53:55.420810462Z INFO: Waiting for ovnk database copies to complete ... 2026-10-30T19:53:55.421258702Z INFO: Copying ovnk databases complete. 2026-10-30T19:53:55.423106236Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory 2026-10-30T19:53:55.423711331Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 699 is not a child of this shell 2026-10-30T19:53:55.423711331Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 700 is not a child of this shell 2026-10-30T19:53:55.423711331Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 701 is not a child of this shell 2026-10-30T19:53:55.423711331Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 702 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 703 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 704 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 705 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 706 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 727 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 728 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 729 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 730 is not a child of this shell 2026-10-30T19:53:55.423740590Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 731 is not a child of this shell 2026-10-30T19:53:55.426933877Z tar: ovnk_database_store: Cannot stat: No such file or directory 2026-10-30T19:53:55.430095212Z tar: Exiting with failure status due to previous errors 2026-10-30T19:53:55.430662684Z INFO: Network log collection complete. 2026-10-30T19:53:55.435031173Z Wrote inspect data to must-gather. 2026-10-30T19:53:55.675861014Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:55.680397429Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-10-30T19:53:55.732791416Z Wrote inspect data to must-gather. 2026-10-30T19:53:55.756733116Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:53:55.817056286Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-10-30T19:53:55.940642607Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:56.009060587Z Wrote inspect data to must-gather. 2026-10-30T19:53:56.244239583Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:56.248207614Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-10-30T19:53:56.398385406Z INFO: Getting status from alertmanager-main-0 2026-10-30T19:53:56.417056822Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod 2026-10-30T19:53:56.552055568Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:56.589248622Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c 2026-10-30T19:53:56.760665627Z daemonset.apps/perf-node-gather-daemonset created 2026-10-30T19:53:56.846912539Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:56.851678748Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-10-30T19:53:56.920534968Z Waiting for performance profile collector pods to become ready: 1 2026-10-30T19:53:57.127905955Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:57.420029122Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:57.423230554Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-10-30T19:53:57.652414547Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:57.937963783Z Wrote inspect data to /must-gather/istio. 2026-10-30T19:53:58.078573977Z Waiting for performance profile collector pods to become ready: 2 2026-10-30T19:53:58.970778146Z Done executing Istio gather script 2026-10-30T19:53:59.224716333Z Waiting for performance profile collector pods to become ready: 3 2026-10-30T19:54:00.381523617Z Waiting for performance profile collector pods to become ready: 4 2026-10-30T19:54:00.761243989Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:00.788158312Z Gathering data for ns/openshift-network-console... 2026-10-30T19:54:01.530222455Z Waiting for performance profile collector pods to become ready: 5 2026-10-30T19:54:01.715588665Z Gathering data for ns/openshift-console-operator... 2026-10-30T19:54:02.550381718Z Gathering data for ns/openshift-console... 2026-10-30T19:54:02.679229334Z Waiting for performance profile collector pods to become ready: 6 2026-10-30T19:54:03.662768401Z Gathering data for ns/openshift-cluster-storage-operator... 2026-10-30T19:54:03.832855812Z Waiting for performance profile collector pods to become ready: 7 2026-10-30T19:54:04.780844021Z Gathering data for ns/openshift-dns-operator... 2026-10-30T19:54:04.992413721Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-10-30T19:54:05.325889721Z Collecting performance related data for node ip-10-0-131-175.ec2.internal 2026-10-30T19:54:05.332726555Z Collecting performance related data for node ip-10-0-143-51.ec2.internal 2026-10-30T19:54:05.340567110Z Collecting performance related data for node ip-10-0-139-153.ec2.internal 2026-10-30T19:54:05.567781596Z Gathering data for ns/openshift-dns... 2026-10-30T19:54:05.765803794Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:07.599714900Z Gathering data for ns/openshift-image-registry... 2026-10-30T19:54:08.949656502Z Gathering data for ns/openshift-ingress-operator... 2026-10-30T19:54:09.733326380Z Gathering data for ns/openshift-ingress... 2026-10-30T19:54:10.600655962Z Gathering data for ns/openshift-ingress-canary... 2026-10-30T19:54:10.770388234Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:11.687017048Z Gathering data for ns/openshift-insights... 2026-10-30T19:54:14.693651323Z Collecting kubelet logs for node ip-10-0-131-175.ec2.internal 2026-10-30T19:54:14.695482713Z Collecting kubelet logs for node ip-10-0-143-51.ec2.internal 2026-10-30T19:54:14.697287490Z Collecting kubelet logs for node ip-10-0-139-153.ec2.internal 2026-10-30T19:54:15.008135947Z Gathering data for ns/openshift-kueue-operator... 2026-10-30T19:54:15.204678188Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-8jw74 namespace 2026-10-30T19:54:15.227026027Z INFO: Node performance data collection complete. 2026-10-30T19:54:15.774917443Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:16.640415381Z Gathering data for ns/openshift-jobset-operator... 2026-10-30T19:54:18.191191057Z Gathering data for ns/openshift-config... 2026-10-30T19:54:18.839657688Z Gathering data for ns/openshift-config-managed... 2026-10-30T19:54:19.618603493Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-10-30T19:54:20.283033503Z Gathering data for ns/openshift-kube-apiserver... 2026-10-30T19:54:20.779703305Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:20.952157355Z Gathering data for ns/openshift-kube-controller-manager... 2026-10-30T19:54:21.674561413Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-10-30T19:54:22.364259859Z Gathering data for ns/openshift-kube-scheduler... 2026-10-30T19:54:23.015076587Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-10-30T19:54:23.783956796Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-10-30T19:54:24.682872182Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-10-30T19:54:25.561009819Z Gathering data for ns/openshift-user-workload-monitoring... 2026-10-30T19:54:25.784230206Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:26.699834984Z Gathering data for ns/openshift-multus... 2026-10-30T19:54:30.788824643Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:32.038383361Z Gathering data for ns/openshift-ovn-kubernetes... 2026-10-30T19:54:35.795022367Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:36.193921746Z Gathering data for ns/openshift-host-network... 2026-10-30T19:54:36.937798034Z Gathering data for ns/openshift-network-diagnostics... 2026-10-30T19:54:38.481269823Z Gathering data for ns/openshift-network-node-identity... 2026-10-30T19:54:39.542825516Z Gathering data for ns/openshift-network-operator... 2026-10-30T19:54:40.583060967Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-10-30T19:54:40.800051360Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:41.210812530Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-10-30T19:54:42.422487143Z Gathering data for ns/openshift-apiserver-operator... 2026-10-30T19:54:43.094446005Z Gathering data for ns/openshift-apiserver... 2026-10-30T19:54:43.987283548Z Gathering data for ns/openshift-controller-manager-operator... 2026-10-30T19:54:44.633066768Z Gathering data for ns/openshift-controller-manager... 2026-10-30T19:54:45.287338228Z Gathering data for ns/openshift-cluster-samples-operator... 2026-10-30T19:54:45.804515595Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:46.611409953Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-10-30T19:54:47.282685143Z Gathering data for ns/openshift-service-ca-operator... 2026-10-30T19:54:48.140372542Z Gathering data for ns/openshift-service-ca... 2026-10-30T19:54:49.082791601Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-10-30T19:54:50.808997401Z [disk usage checker] Volume usage percentage: current = 26 ; allowed = 70 2026-10-30T19:54:51.294179899Z Wrote inspect data to must-gather. 2026-10-30T19:54:51.294223628Z error: inspection completed with the errors occurred while gathering data: 2026-10-30T19:54:51.294223628Z skipping gathering secrets/support due to error: secrets "support" not found 2026-10-30T19:54:51.504034072Z error: the server doesn't have a resource type "clusters" 2026-10-30T19:54:51.588131124Z Caches written to disk