2026-09-17T17:34:26.554915582Z [disk usage checker] Started 2026-09-17T17:34:26.558388158Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70 2026-09-17T17:34:26.907752276Z Gathering data for ns/openshift-cluster-version... 2026-09-17T17:34:27.045599378Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T17:34:27.151141848Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T17:34:27.234571888Z Gathering data for ns/default... 2026-09-17T17:34:27.564510151Z Gathering data for ns/openshift... 2026-09-17T17:34:27.994667938Z Gathering data for ns/kube-system... 2026-09-17T17:34:28.741907156Z Gathering data for ns/openshift-etcd... 2026-09-17T17:34:29.045698871Z Wrote inspect data to must-gather. 2026-09-17T17:34:29.045730321Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T17:34:29.045730321Z namespaces "assisted-installer" not found 2026-09-17T17:34:29.478698522Z Waiting on subprocesses to finish execution. 2026-09-17T17:34:29.482921214Z INFO: Gathering HAProxy config files 2026-09-17T17:34:29.486061217Z Executing Istio gather script 2026-09-17T17:34:29.486765870Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-17T17:34:29.488452508Z INFO: Collecting host service logs for crio 2026-09-17T17:34:29.488756616Z INFO: Collecting host service logs for kubelet 2026-09-17T17:34:29.489091039Z INFO: Collecting host service logs for rpm-ostreed 2026-09-17T17:34:29.489471295Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-17T17:34:29.489942835Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-17T17:34:29.490317268Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-17T17:34:29.490647624Z INFO: Collecting host service logs for NetworkManager 2026-09-17T17:34:29.490971603Z INFO: Collecting host service logs for openvswitch 2026-09-17T17:34:29.491316230Z INFO: Collecting host service logs for ovs-configuration 2026-09-17T17:34:29.491654886Z INFO: Collecting host service logs for ovsdb-server 2026-09-17T17:34:29.492020127Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-17T17:34:29.492706716Z INFO: Waiting for worker host service log collection to complete ... 2026-09-17T17:34:29.498336309Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-17T17:34:29.511299537Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-17T17:34:29.732584028Z INFO: Waiting for node performance related collection to complete ... 2026-09-17T17:34:30.104996102Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-17T17:34:30.105033895Z INFO: on-disk MachineConfig config collection complete. 2026-09-17T17:34:30.136501570Z Wrote inspect data to must-gather. 2026-09-17T17:34:30.166561002Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-17T17:34:30.567054684Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-17T17:34:30.750927121Z No resources found in openshift-etcd namespace. 2026-09-17T17:34:30.769758344Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-17T17:34:30.772937701Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-17T17:34:30.808550306Z INFO: Collecting Insights Archives from insights-operator-6f9c644dd9-678qx 2026-09-17T17:34:30.808601559Z insights-runtime-extractor-4zxsg 2026-09-17T17:34:30.808634913Z insights-runtime-extractor-jmjx7 2026-09-17T17:34:30.808656331Z insights-runtime-extractor-qzw8z 2026-09-17T17:34:30.871593147Z INFO: "metallb-operator" not detected. Skipping. 2026-09-17T17:34:30.958622038Z INFO: Worker host service log collection to complete. 2026-09-17T17:34:30.979016199Z error: the server doesn't have a resource type "performanceprofile" 2026-09-17T17:34:31.009899471Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-17T17:34:31.148224640Z No resources found 2026-09-17T17:34:31.195179107Z ERROR: No running kube-apiserver pods found 2026-09-17T17:34:31.532930464Z INFO: 'previous-logs' folder not found on ip-10-0-131-202.ec2.internal, skipping... 2026-09-17T17:34:31.582152766Z No resources found 2026-09-17T17:34:31.593615913Z Wrote inspect data to must-gather. 2026-09-17T17:34:31.645639796Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70 2026-09-17T17:34:31.647346069Z Wrote inspect data to must-gather. 2026-09-17T17:34:31.716401149Z Wrote inspect data to must-gather. 2026-09-17T17:34:31.829347778Z Wrote inspect data to must-gather. 2026-09-17T17:34:31.868448191Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-17T17:34:31.903218894Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-17T17:34:31.903218894Z See 'oc rsync -h' for help and examples 2026-09-17T17:34:31.928175339Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T17:34:31.995526680Z Wrote inspect data to must-gather. 2026-09-17T17:34:32.018840985Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-17T17:34:32.064522931Z tar: Removing leading `/' from member names 2026-09-17T17:34:32.072301940Z INFO: HAProxy config collection complete. 2026-09-17T17:34:32.153869710Z Wrote inspect data to must-gather. 2026-09-17T17:34:32.189192052Z INFO: 'previous-logs' folder not found on ip-10-0-135-124.ec2.internal, skipping... 2026-09-17T17:34:32.273403226Z Wrote inspect data to must-gather. 2026-09-17T17:34:32.365974343Z error: the server doesn't have a resource type "machineconfigs" 2026-09-17T17:34:32.440276710Z Gathering data for ns/openshift-monitoring... 2026-09-17T17:34:32.516468054Z INFO: 'previous-logs' folder not found on ip-10-0-138-129.ec2.internal, skipping... 2026-09-17T17:34:32.516502566Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-17T17:34:32.516502566Z INFO: Machine Config Daemon termination log collection complete. 2026-09-17T17:34:32.531646283Z INFO: Getting rules from prometheus-k8s-0 2026-09-17T17:34:32.587148080Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-17T17:34:32.621340999Z Wrote inspect data to must-gather. 2026-09-17T17:34:32.833306330Z Wrote inspect data to must-gather. 2026-09-17T17:34:32.904158012Z Wrote inspect data to must-gather. 2026-09-17T17:34:33.029528839Z INFO: Getting status/config from prometheus-k8s-0 2026-09-17T17:34:33.035965503Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-17T17:34:33.128683085Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-17T17:34:33.241689792Z INFO: INTERCONNECT MODE 2026-09-17T17:34:33.241714978Z INFO: Gathering ovn-kubernetes DBs 2026-09-17T17:34:33.268902814Z Wrote inspect data to must-gather. 2026-09-17T17:34:33.431368181Z INFO: Gathering OVN_Northbound from ovnkube-node-2w4sg... 2026-09-17T17:34:33.431622028Z INFO: Gathering OVN_Northbound from ovnkube-node-6mnll... 2026-09-17T17:34:33.431868588Z INFO: Gathering OVN_Northbound from ovnkube-node-cxtcx... 2026-09-17T17:34:33.432140008Z INFO: Gathering OVN_Southbound from ovnkube-node-2w4sg... 2026-09-17T17:34:33.432402322Z INFO: Gathering OVN_Southbound from ovnkube-node-6mnll... 2026-09-17T17:34:33.432646286Z INFO: Gathering OVN_Southbound from ovnkube-node-cxtcx... 2026-09-17T17:34:33.550520093Z Wrote inspect data to must-gather. 2026-09-17T17:34:33.569335448Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-17T17:34:33.743726149Z tar: Removing leading `/' from member names 2026-09-17T17:34:33.753436476Z tar: Removing leading `/' from member names 2026-09-17T17:34:33.764625926Z tar: Removing leading `/' from member names 2026-09-17T17:34:33.783645066Z tar: Removing leading `/' from member names 2026-09-17T17:34:33.784451688Z tar: Removing leading `/' from member names 2026-09-17T17:34:33.795064493Z tar: Removing leading `/' from member names 2026-09-17T17:34:33.869626083Z Wrote inspect data to must-gather. 2026-09-17T17:34:33.913166079Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-17T17:34:34.141117318Z Wrote inspect data to must-gather. 2026-09-17T17:34:34.211376852Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-17T17:34:34.236050065Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-17T17:34:34.315824528Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-17T17:34:34.362448642Z INFO: Waiting for network log collection to complete ... 2026-09-17T17:34:34.362501180Z INFO: Waiting for ovnk database copies to complete ... 2026-09-17T17:34:34.363031705Z INFO: Copying ovnk databases complete. 2026-09-17T17:34:34.365164546Z 4.7M must-gather/network_logs/ovnk_database_store 2026-09-17T17:34:34.367718288Z ovnk_database_store/ 2026-09-17T17:34:34.367718288Z ovnk_database_store/ovnkube-node-cxtcx_sbdb 2026-09-17T17:34:34.384798887Z Wrote inspect data to must-gather. 2026-09-17T17:34:34.395744062Z ovnk_database_store/ovnkube-node-2w4sg_nbdb 2026-09-17T17:34:34.405773815Z ovnk_database_store/ovnkube-node-cxtcx_nbdb 2026-09-17T17:34:34.419178623Z ovnk_database_store/ovnkube-node-6mnll_sbdb 2026-09-17T17:34:34.440739882Z ovnk_database_store/ovnkube-node-6mnll_nbdb 2026-09-17T17:34:34.453628071Z ovnk_database_store/ovnkube-node-2w4sg_sbdb 2026-09-17T17:34:34.456103488Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-17T17:34:34.490672657Z INFO: Network log collection complete. 2026-09-17T17:34:34.531921517Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:34.625986339Z Wrote inspect data to must-gather. 2026-09-17T17:34:34.756621111Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:34.759705093Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-17T17:34:34.845962061Z W0917 17:34:34.845921 1525 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-09-17T17:34:34.850805094Z Wrote inspect data to must-gather. 2026-09-17T17:34:34.887068626Z INFO: Getting status from alertmanager-main-0 2026-09-17T17:34:34.894680608Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-17T17:34:34.986207936Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:35.096759769Z Gathering data for ns/openshift-network-console... 2026-09-17T17:34:35.169696702Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-17T17:34:35.175799582Z INFO: Fallback to identify the container image from release info 2026-09-17T17:34:35.220166314Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:35.224169140Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-17T17:34:35.416193547Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:35.536548790Z Gathering data for ns/openshift-console-operator... 2026-09-17T17:34:35.619540718Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:35.623485269Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-17T17:34:35.824690353Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0720c8b078cc0cebb01b4c5cde21180aee7c2bb1a0d322b21890c0d3707406ea 2026-09-17T17:34:35.849587576Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:35.976022676Z Gathering data for ns/openshift-console... 2026-09-17T17:34:35.976428285Z daemonset.apps/perf-node-gather-daemonset created 2026-09-17T17:34:36.057535586Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:36.061215306Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-09-17T17:34:36.122023305Z Waiting for performance profile collector pods to become ready: 1 2026-09-17T17:34:36.239553939Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:36.463514215Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:36.466736513Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-09-17T17:34:36.556958196Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-17T17:34:36.650706565Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:36.662376881Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70 2026-09-17T17:34:36.844813350Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:36.850608145Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-17T17:34:37.029639162Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:37.088488114Z Gathering data for ns/openshift-dns-operator... 2026-09-17T17:34:37.236804939Z Wrote inspect data to /must-gather/istio. 2026-09-17T17:34:37.270243390Z Waiting for performance profile collector pods to become ready: 2 2026-09-17T17:34:37.429535500Z Gathering data for ns/openshift-dns... 2026-09-17T17:34:37.940033699Z Done executing Istio gather script 2026-09-17T17:34:38.030027667Z Gathering data for ns/openshift-image-registry... 2026-09-17T17:34:38.398041990Z Waiting for performance profile collector pods to become ready: 3 2026-09-17T17:34:38.510833126Z Gathering data for ns/openshift-ingress-operator... 2026-09-17T17:34:38.848824490Z Gathering data for ns/openshift-ingress... 2026-09-17T17:34:39.219014658Z Gathering data for ns/openshift-ingress-canary... 2026-09-17T17:34:39.536267487Z Waiting for performance profile collector pods to become ready: 4 2026-09-17T17:34:39.606414279Z Gathering data for ns/openshift-insights... 2026-09-17T17:34:40.650414773Z Waiting for performance profile collector pods to become ready: 5 2026-09-17T17:34:41.667854712Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70 2026-09-17T17:34:41.783265645Z Waiting for performance profile collector pods to become ready: 6 2026-09-17T17:34:41.952121500Z Gathering data for ns/kserve... 2026-09-17T17:34:42.912540777Z Waiting for performance profile collector pods to become ready: 7 2026-09-17T17:34:44.046921576Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-17T17:34:44.299625171Z Collecting performance related data for node ip-10-0-138-129.ec2.internal 2026-09-17T17:34:44.306289364Z Collecting performance related data for node ip-10-0-131-202.ec2.internal 2026-09-17T17:34:44.314069331Z Collecting performance related data for node ip-10-0-135-124.ec2.internal 2026-09-17T17:34:44.499772758Z Gathering data for ns/openshift-config... 2026-09-17T17:34:44.855608769Z Gathering data for ns/openshift-config-managed... 2026-09-17T17:34:45.489051762Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-17T17:34:45.861516141Z Gathering data for ns/openshift-kube-apiserver... 2026-09-17T17:34:46.185218491Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-17T17:34:46.538068680Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-17T17:34:46.672328559Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70 2026-09-17T17:34:46.876226588Z Gathering data for ns/openshift-kube-scheduler... 2026-09-17T17:34:47.270626964Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-17T17:34:47.620272522Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-17T17:34:47.992348430Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-17T17:34:48.328988770Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-17T17:34:48.893071260Z Gathering data for ns/openshift-multus... 2026-09-17T17:34:50.388175007Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-17T17:34:51.677989223Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70 2026-09-17T17:34:51.712622297Z Gathering data for ns/openshift-host-network... 2026-09-17T17:34:52.036060583Z Gathering data for ns/openshift-network-diagnostics... 2026-09-17T17:34:52.580326429Z Gathering data for ns/openshift-network-node-identity... 2026-09-17T17:34:52.738289385Z Collecting kubelet logs for node ip-10-0-138-129.ec2.internal 2026-09-17T17:34:52.740156433Z Collecting kubelet logs for node ip-10-0-131-202.ec2.internal 2026-09-17T17:34:52.741899634Z Collecting kubelet logs for node ip-10-0-135-124.ec2.internal 2026-09-17T17:34:53.079416440Z Gathering data for ns/openshift-network-operator... 2026-09-17T17:34:53.213687969Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-lrk88 namespace 2026-09-17T17:34:53.219840188Z INFO: Node performance data collection complete. 2026-09-17T17:34:53.485449714Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-17T17:34:53.815585520Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-17T17:34:54.319339361Z Gathering data for ns/openshift-apiserver-operator... 2026-09-17T17:34:54.660793939Z Gathering data for ns/openshift-apiserver... 2026-09-17T17:34:55.094425844Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-17T17:34:55.410403053Z Gathering data for ns/openshift-controller-manager... 2026-09-17T17:34:55.699173787Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-17T17:34:56.376178229Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-17T17:34:56.683063490Z [disk usage checker] Volume usage percentage: current = 36 ; allowed = 70 2026-09-17T17:34:56.686079521Z Gathering data for ns/openshift-service-ca-operator... 2026-09-17T17:34:57.018064319Z Gathering data for ns/openshift-service-ca... 2026-09-17T17:34:57.395052466Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-17T17:34:58.261207893Z Wrote inspect data to must-gather. 2026-09-17T17:34:58.261269442Z error: inspection completed with the errors occurred while gathering data: 2026-09-17T17:34:58.261269442Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-17T17:34:58.392668736Z error: the server doesn't have a resource type "clusters" 2026-09-17T17:34:58.545103040Z Caches written to disk