2026-06-09T16:05:30.370653842Z [disk usage checker] Started 2026-06-09T16:05:30.374436565Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-06-09T16:05:30.568326546Z Gathering data for ns/openshift-cluster-version... 2026-06-09T16:05:30.709252524Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-09T16:05:30.787859207Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-09T16:05:30.859362326Z Gathering data for ns/default... 2026-06-09T16:05:31.128713695Z Gathering data for ns/openshift... 2026-06-09T16:05:31.508196892Z Gathering data for ns/kube-system... 2026-06-09T16:05:32.153417943Z Gathering data for ns/openshift-etcd... 2026-06-09T16:05:32.430107190Z Wrote inspect data to must-gather. 2026-06-09T16:05:32.430135532Z error: inspection completed with the errors occurred while gathering data: 2026-06-09T16:05:32.430135532Z namespaces "assisted-installer" not found 2026-06-09T16:05:32.924486706Z Waiting on subprocesses to finish execution. 2026-06-09T16:05:32.932288052Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-06-09T16:05:32.937567456Z INFO: Gathering HAProxy config files 2026-06-09T16:05:32.938676339Z Executing Istio gather script 2026-06-09T16:05:32.941264725Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-06-09T16:05:32.947360709Z INFO: Waiting for node performance related collection to complete ... 2026-06-09T16:05:32.958972074Z INFO: Gathering machine config daemon's old logs from all nodes 2026-06-09T16:05:32.977995893Z INFO: Collecting host service logs for crio 2026-06-09T16:05:32.978359261Z INFO: Collecting host service logs for kubelet 2026-06-09T16:05:32.978711129Z INFO: Collecting host service logs for rpm-ostreed 2026-06-09T16:05:32.979074828Z INFO: Collecting host service logs for ostree-finalize-staged 2026-06-09T16:05:32.979811414Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-06-09T16:05:32.980873601Z INFO: Collecting host service logs for machine-config-daemon-host 2026-06-09T16:05:32.981211948Z INFO: Collecting host service logs for NetworkManager 2026-06-09T16:05:32.981517786Z INFO: Collecting host service logs for openvswitch 2026-06-09T16:05:32.981818251Z INFO: Collecting host service logs for ovs-configuration 2026-06-09T16:05:32.982159154Z INFO: Collecting host service logs for ovsdb-server 2026-06-09T16:05:32.982469914Z INFO: Collecting host service logs for ovs-vswitchd 2026-06-09T16:05:32.982828459Z INFO: Waiting for worker host service log collection to complete ... 2026-06-09T16:05:33.386158193Z Wrote inspect data to must-gather. 2026-06-09T16:05:33.769695634Z INFO: Collecting Insights Archives from insights-operator-896549976-k9g5s 2026-06-09T16:05:33.769749033Z insights-runtime-extractor-6z7np 2026-06-09T16:05:33.769768146Z insights-runtime-extractor-fgw42 2026-06-09T16:05:33.769786648Z insights-runtime-extractor-frn75 2026-06-09T16:05:33.900220375Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-06-09T16:05:34.013670534Z INFO: "sriov-network-operator" not detected. Skipping. 2026-06-09T16:05:34.016187791Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-06-09T16:05:34.029952129Z INFO: Found 1 replicas - prometheus-k8s-0 2026-06-09T16:05:34.154749944Z error: the server doesn't have a resource type "performanceprofile" 2026-06-09T16:05:34.176010516Z No resources found in openshift-etcd namespace. 2026-06-09T16:05:34.218351299Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-06-09T16:05:34.218390656Z INFO: on-disk MachineConfig config collection complete. 2026-06-09T16:05:34.301424944Z No resources found 2026-06-09T16:05:34.390845819Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-06-09T16:05:34.390845819Z See 'oc rsync -h' for help and examples 2026-06-09T16:05:34.602910394Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-06-09T16:05:34.686979938Z INFO: "metallb-operator" not detected. Skipping. 2026-06-09T16:05:34.758676905Z INFO: Worker host service log collection to complete. 2026-06-09T16:05:34.772160701Z INFO: 'previous-logs' folder not found on ip-10-0-129-90.ec2.internal, skipping... 2026-06-09T16:05:34.806904457Z ERROR: No running kube-apiserver pods found 2026-06-09T16:05:34.881544026Z No resources found 2026-06-09T16:05:35.024070308Z Gathering data for ns/openshift-monitoring... 2026-06-09T16:05:35.036935448Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.057746521Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.137339206Z INFO: Waiting for HAProxy config collection to complete ... 2026-06-09T16:05:35.318985707Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.326013957Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-06-09T16:05:35.402639425Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-06-09T16:05:35.410005041Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-06-09T16:05:35.449948000Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.456671704Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-09T16:05:35.478222830Z tar: Removing leading `/' from member names 2026-06-09T16:05:35.485307750Z INFO: HAProxy config collection complete. 2026-06-09T16:05:35.515438667Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.528976739Z error: the server doesn't have a resource type "machineconfigs" 2026-06-09T16:05:35.539620591Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.555204137Z INFO: Getting rules from prometheus-k8s-0 2026-06-09T16:05:35.573332846Z INFO: 'previous-logs' folder not found on ip-10-0-138-11.ec2.internal, skipping... 2026-06-09T16:05:35.699971275Z error: the server doesn't have a resource type "machineconfigpools" 2026-06-09T16:05:35.805021740Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.843621571Z INFO: 'previous-logs' folder not found on ip-10-0-139-195.ec2.internal, skipping... 2026-06-09T16:05:35.843664398Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-06-09T16:05:35.843664398Z INFO: Machine Config Daemon termination log collection complete. 2026-06-09T16:05:35.942786611Z Wrote inspect data to must-gather. 2026-06-09T16:05:35.971236600Z INFO: Getting status/config from prometheus-k8s-0 2026-06-09T16:05:36.068355890Z Wrote inspect data to must-gather. 2026-06-09T16:05:36.111907834Z error: the server doesn't have a resource type "kubeletconfigs" 2026-06-09T16:05:36.315018117Z Wrote inspect data to must-gather. 2026-06-09T16:05:36.318369695Z Wrote inspect data to must-gather. 2026-06-09T16:05:36.363837787Z INFO: Getting status/flags from prometheus-k8s-0 2026-06-09T16:05:36.487603213Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-06-09T16:05:36.547573040Z Wrote inspect data to must-gather. 2026-06-09T16:05:36.609376797Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-06-09T16:05:36.629606623Z INFO: INTERCONNECT MODE 2026-06-09T16:05:36.629703050Z INFO: Gathering ovn-kubernetes DBs 2026-06-09T16:05:36.658050432Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-06-09T16:05:36.702899467Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:36.760722664Z Wrote inspect data to must-gather. 2026-06-09T16:05:36.795566502Z INFO: Gathering OVN_Northbound from ovnkube-node-776ww... 2026-06-09T16:05:36.795797713Z INFO: Gathering OVN_Northbound from ovnkube-node-bgvmw... 2026-06-09T16:05:36.796050673Z INFO: Gathering OVN_Northbound from ovnkube-node-wqndd... 2026-06-09T16:05:36.796329343Z INFO: Gathering OVN_Southbound from ovnkube-node-776ww... 2026-06-09T16:05:36.796636246Z INFO: Gathering OVN_Southbound from ovnkube-node-bgvmw... 2026-06-09T16:05:36.796928531Z INFO: Gathering OVN_Southbound from ovnkube-node-wqndd... 2026-06-09T16:05:36.916382854Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-06-09T16:05:37.080138764Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:37.082617018Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-06-09T16:05:37.085216729Z tar: Removing leading `/' from member names 2026-06-09T16:05:37.104389526Z Wrote inspect data to must-gather. 2026-06-09T16:05:37.139737115Z tar: Removing leading `/' from member names 2026-06-09T16:05:37.155208592Z tar: Removing leading `/' from member names 2026-06-09T16:05:37.172654973Z tar: Removing leading `/' from member names 2026-06-09T16:05:37.203060712Z tar: Removing leading `/' from member names 2026-06-09T16:05:37.206390572Z tar: Removing leading `/' from member names 2026-06-09T16:05:37.300859778Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-06-09T16:05:37.319792197Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:37.342570329Z W0609 16:05:37.342516 1425 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-06-09T16:05:37.349658031Z Wrote inspect data to must-gather. 2026-06-09T16:05:37.486255885Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-06-09T16:05:37.567773261Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:37.570915554Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-06-09T16:05:37.673517128Z INFO: Waiting for network log collection to complete ... 2026-06-09T16:05:37.673557559Z INFO: Waiting for ovnk database copies to complete ... 2026-06-09T16:05:37.674073189Z INFO: Copying ovnk databases complete. 2026-06-09T16:05:37.676280749Z 2.5M must-gather/network_logs/ovnk_database_store 2026-06-09T16:05:37.679473353Z ovnk_database_store/ 2026-06-09T16:05:37.679526064Z ovnk_database_store/ovnkube-node-wqndd_sbdb 2026-06-09T16:05:37.692154929Z Gathering data for ns/openshift-network-console... 2026-06-09T16:05:37.696369052Z ovnk_database_store/ovnkube-node-wqndd_nbdb 2026-06-09T16:05:37.703484065Z ovnk_database_store/ovnkube-node-bgvmw_sbdb 2026-06-09T16:05:37.716568939Z Wrote inspect data to must-gather. 2026-06-09T16:05:37.720429082Z ovnk_database_store/ovnkube-node-776ww_sbdb 2026-06-09T16:05:37.739209792Z ovnk_database_store/ovnkube-node-bgvmw_nbdb 2026-06-09T16:05:37.746839310Z ovnk_database_store/ovnkube-node-776ww_nbdb 2026-06-09T16:05:37.758752353Z INFO: Network log collection complete. 2026-06-09T16:05:37.759831666Z INFO: Getting status from alertmanager-main-0 2026-06-09T16:05:37.805642153Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:38.015448168Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:38.017571238Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-06-09T16:05:38.044434624Z Wrote inspect data to must-gather. 2026-06-09T16:05:38.128276161Z Gathering data for ns/openshift-console-operator... 2026-06-09T16:05:38.251736539Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:38.302606764Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-06-09T16:05:38.304705026Z INFO: Fallback to identify the container image from release info 2026-06-09T16:05:38.441592035Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:38.443930978Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-06-09T16:05:38.537389255Z Gathering data for ns/openshift-console... 2026-06-09T16:05:38.600425537Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:38.782768393Z Wrote inspect data to /must-gather/istio. 2026-06-09T16:05:39.018455924Z Gathering data for ns/openshift-cluster-storage-operator... 2026-06-09T16:05:39.028633177Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e 2026-06-09T16:05:39.166414042Z daemonset.apps/perf-node-gather-daemonset created 2026-06-09T16:05:39.303691451Z Waiting for performance profile collector pods to become ready: 1 2026-06-09T16:05:39.493317346Z Done executing Istio gather script 2026-06-09T16:05:39.533162201Z Gathering data for ns/openshift-dns-operator... 2026-06-09T16:05:39.804298793Z Gathering data for ns/openshift-dns... 2026-06-09T16:05:40.355848833Z Gathering data for ns/openshift-image-registry... 2026-06-09T16:05:40.409397335Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-06-09T16:05:40.413854539Z Waiting for performance profile collector pods to become ready: 2 2026-06-09T16:05:40.772639781Z Gathering data for ns/openshift-ingress-operator... 2026-06-09T16:05:41.054321406Z Gathering data for ns/openshift-ingress... 2026-06-09T16:05:41.360268010Z Gathering data for ns/openshift-ingress-canary... 2026-06-09T16:05:41.534563909Z Waiting for performance profile collector pods to become ready: 3 2026-06-09T16:05:41.718807068Z Gathering data for ns/openshift-insights... 2026-06-09T16:05:42.647792681Z Waiting for performance profile collector pods to become ready: 4 2026-06-09T16:05:43.510324205Z Gathering data for ns/openshift-config... 2026-06-09T16:05:43.750828297Z Gathering data for ns/openshift-config-managed... 2026-06-09T16:05:43.773367316Z Waiting for performance profile collector pods to become ready: 5 2026-06-09T16:05:44.140533000Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-06-09T16:05:44.418789001Z Gathering data for ns/openshift-kube-apiserver... 2026-06-09T16:05:44.646488086Z Gathering data for ns/openshift-kube-controller-manager... 2026-06-09T16:05:44.888329169Z Waiting for performance profile collector pods to become ready: 6 2026-06-09T16:05:44.912168077Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-06-09T16:05:45.161357012Z Gathering data for ns/openshift-kube-scheduler... 2026-06-09T16:05:45.380990302Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-06-09T16:05:45.415439795Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-06-09T16:05:45.640175884Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-06-09T16:05:45.921787943Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-06-09T16:05:46.012393471Z Waiting for performance profile collector pods to become ready: 7 2026-06-09T16:05:46.209879030Z Gathering data for ns/openshift-user-workload-monitoring... 2026-06-09T16:05:46.650132522Z Gathering data for ns/openshift-multus... 2026-06-09T16:05:47.114960079Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-06-09T16:05:47.334318723Z Collecting performance related data for node ip-10-0-139-195.ec2.internal 2026-06-09T16:05:47.341135840Z Collecting performance related data for node ip-10-0-129-90.ec2.internal 2026-06-09T16:05:47.348076831Z Collecting performance related data for node ip-10-0-138-11.ec2.internal 2026-06-09T16:05:48.141466958Z Gathering data for ns/openshift-ovn-kubernetes... 2026-06-09T16:05:49.340621726Z Gathering data for ns/openshift-host-network... 2026-06-09T16:05:49.664318313Z Gathering data for ns/openshift-network-diagnostics... 2026-06-09T16:05:50.196682035Z Gathering data for ns/openshift-network-node-identity... 2026-06-09T16:05:50.422417994Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-06-09T16:05:50.607574606Z Gathering data for ns/openshift-network-operator... 2026-06-09T16:05:50.948346485Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-06-09T16:05:51.194004305Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-06-09T16:05:51.644094308Z Gathering data for ns/openshift-apiserver-operator... 2026-06-09T16:05:51.961451715Z Gathering data for ns/openshift-apiserver... 2026-06-09T16:05:52.370467858Z Gathering data for ns/openshift-controller-manager-operator... 2026-06-09T16:05:52.623017790Z Gathering data for ns/openshift-controller-manager... 2026-06-09T16:05:52.891060484Z Gathering data for ns/openshift-cluster-samples-operator... 2026-06-09T16:05:53.539067340Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-06-09T16:05:53.795630486Z Gathering data for ns/openshift-service-ca-operator... 2026-06-09T16:05:54.081015994Z Gathering data for ns/openshift-service-ca... 2026-06-09T16:05:54.421506928Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-06-09T16:05:55.287142197Z Wrote inspect data to must-gather. 2026-06-09T16:05:55.287188256Z error: inspection completed with the errors occurred while gathering data: 2026-06-09T16:05:55.287188256Z skipping gathering secrets/support due to error: secrets "support" not found 2026-06-09T16:05:55.374333029Z Collecting kubelet logs for node ip-10-0-139-195.ec2.internal 2026-06-09T16:05:55.376060461Z Collecting kubelet logs for node ip-10-0-129-90.ec2.internal 2026-06-09T16:05:55.377772483Z Collecting kubelet logs for node ip-10-0-138-11.ec2.internal 2026-06-09T16:05:55.428617267Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70 2026-06-09T16:05:55.663642462Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ktdb2 namespace 2026-06-09T16:05:55.669005001Z INFO: Node performance data collection complete. 2026-06-09T16:05:55.795347248Z error: the server doesn't have a resource type "clusters" 2026-06-09T16:05:55.917556264Z Caches written to disk