2026-07-24T07:10:37.829177171Z [disk usage checker] Started 2026-07-24T07:10:37.832525847Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-07-24T07:10:38.045195661Z Gathering data for ns/openshift-cluster-version... 2026-07-24T07:10:38.188682662Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-24T07:10:38.303163622Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-24T07:10:38.385559891Z Gathering data for ns/default... 2026-07-24T07:10:38.760942170Z Gathering data for ns/openshift... 2026-07-24T07:10:39.149239201Z Gathering data for ns/kube-system... 2026-07-24T07:10:39.828642849Z Gathering data for ns/openshift-etcd... 2026-07-24T07:10:40.109106998Z Wrote inspect data to must-gather. 2026-07-24T07:10:40.109137443Z error: inspection completed with the errors occurred while gathering data: 2026-07-24T07:10:40.109137443Z namespaces "assisted-installer" not found 2026-07-24T07:10:40.334055585Z Waiting on subprocesses to finish execution. 2026-07-24T07:10:40.343979402Z Executing Istio gather script 2026-07-24T07:10:40.346857536Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-24T07:10:40.347946416Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-24T07:10:40.357358982Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-24T07:10:40.364195488Z INFO: Collecting host service logs for crio 2026-07-24T07:10:40.364488142Z INFO: Collecting host service logs for kubelet 2026-07-24T07:10:40.364788744Z INFO: Collecting host service logs for rpm-ostreed 2026-07-24T07:10:40.365086668Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-24T07:10:40.365395245Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-24T07:10:40.365686232Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-24T07:10:40.365974166Z INFO: Collecting host service logs for NetworkManager 2026-07-24T07:10:40.366069595Z INFO: Gathering HAProxy config files 2026-07-24T07:10:40.366255996Z INFO: Collecting host service logs for openvswitch 2026-07-24T07:10:40.366565037Z INFO: Collecting host service logs for ovs-configuration 2026-07-24T07:10:40.366917350Z INFO: Collecting host service logs for ovsdb-server 2026-07-24T07:10:40.367251005Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-24T07:10:40.367661338Z INFO: Waiting for worker host service log collection to complete ... 2026-07-24T07:10:40.497772704Z INFO: Waiting for node performance related collection to complete ... 2026-07-24T07:10:40.721956388Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-zfr9b 2026-07-24T07:10:40.722036546Z insights-runtime-extractor-qcht8 2026-07-24T07:10:40.722056995Z insights-runtime-extractor-tnfrg 2026-07-24T07:10:40.722075959Z insights-runtime-extractor-tp6ss 2026-07-24T07:10:40.830265189Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-24T07:10:41.034095658Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-24T07:10:41.061615877Z No resources found in openshift-etcd namespace. 2026-07-24T07:10:41.150872945Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-24T07:10:41.150961108Z INFO: on-disk MachineConfig config collection complete. 2026-07-24T07:10:41.307011691Z No resources found 2026-07-24T07:10:41.327520645Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-24T07:10:41.493839663Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-24T07:10:41.493839663Z See 'oc rsync -h' for help and examples 2026-07-24T07:10:41.576881983Z INFO: Worker host service log collection to complete. 2026-07-24T07:10:41.602844530Z error: the server doesn't have a resource type "performanceprofile" 2026-07-24T07:10:41.604618523Z INFO: "metallb-operator" not detected. Skipping. 2026-07-24T07:10:41.619587980Z Wrote inspect data to must-gather. 2026-07-24T07:10:41.750710068Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-24T07:10:41.890718853Z ERROR: No running kube-apiserver pods found 2026-07-24T07:10:41.917057566Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-24T07:10:42.055045248Z Wrote inspect data to must-gather. 2026-07-24T07:10:42.141095323Z Wrote inspect data to must-gather. 2026-07-24T07:10:42.246114498Z INFO: 'previous-logs' folder not found on ip-10-0-128-250.ec2.internal, skipping... 2026-07-24T07:10:42.336422952Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-24T07:10:42.573704522Z Gathering data for ns/openshift-monitoring... 2026-07-24T07:10:42.586877635Z error: the server doesn't have a resource type "machineconfigs" 2026-07-24T07:10:42.691732117Z Wrote inspect data to must-gather. 2026-07-24T07:10:42.697200731Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-24T07:10:42.737966506Z No resources found 2026-07-24T07:10:42.852535923Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-24T07:10:42.866169792Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-07-24T07:10:42.927509558Z tar: Removing leading `/' from member names 2026-07-24T07:10:42.936852068Z INFO: HAProxy config collection complete. 2026-07-24T07:10:42.982015816Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-24T07:10:42.984860374Z Wrote inspect data to must-gather. 2026-07-24T07:10:42.989238843Z Wrote inspect data to must-gather. 2026-07-24T07:10:43.035294420Z Wrote inspect data to must-gather. 2026-07-24T07:10:43.064113019Z INFO: 'previous-logs' folder not found on ip-10-0-135-109.ec2.internal, skipping... 2026-07-24T07:10:43.244927923Z Wrote inspect data to must-gather. 2026-07-24T07:10:43.258890950Z INFO: Getting rules from prometheus-k8s-0 2026-07-24T07:10:43.304010133Z Wrote inspect data to must-gather. 2026-07-24T07:10:43.364517527Z INFO: 'previous-logs' folder not found on ip-10-0-138-219.ec2.internal, skipping... 2026-07-24T07:10:43.364555372Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-24T07:10:43.364589265Z INFO: Machine Config Daemon termination log collection complete. 2026-07-24T07:10:43.429134763Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-24T07:10:43.444947931Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-24T07:10:43.563077020Z Wrote inspect data to must-gather. 2026-07-24T07:10:43.659439493Z Wrote inspect data to must-gather. 2026-07-24T07:10:43.690030053Z INFO: Getting status/config from prometheus-k8s-0 2026-07-24T07:10:43.814040340Z Wrote inspect data to must-gather. 2026-07-24T07:10:43.867584281Z Wrote inspect data to must-gather. 2026-07-24T07:10:44.095453494Z Wrote inspect data to must-gather. 2026-07-24T07:10:44.109364580Z INFO: INTERCONNECT MODE 2026-07-24T07:10:44.109403102Z INFO: Gathering ovn-kubernetes DBs 2026-07-24T07:10:44.110081569Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-24T07:10:44.196200929Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-24T07:10:44.265500603Z INFO: Gathering OVN_Northbound from ovnkube-node-77hzx... 2026-07-24T07:10:44.265774059Z INFO: Gathering OVN_Northbound from ovnkube-node-rcqc6... 2026-07-24T07:10:44.266008526Z INFO: Gathering OVN_Northbound from ovnkube-node-xls2w... 2026-07-24T07:10:44.266321953Z INFO: Gathering OVN_Southbound from ovnkube-node-77hzx... 2026-07-24T07:10:44.266599212Z INFO: Gathering OVN_Southbound from ovnkube-node-rcqc6... 2026-07-24T07:10:44.266873186Z INFO: Gathering OVN_Southbound from ovnkube-node-xls2w... 2026-07-24T07:10:44.297021275Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-24T07:10:44.351764747Z Wrote inspect data to must-gather. 2026-07-24T07:10:44.378328275Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-24T07:10:44.573093637Z tar: Removing leading `/' from member names 2026-07-24T07:10:44.599223795Z tar: Removing leading `/' from member names 2026-07-24T07:10:44.615739459Z tar: Removing leading `/' from member names 2026-07-24T07:10:44.625396792Z tar: Removing leading `/' from member names 2026-07-24T07:10:44.625572247Z tar: Removing leading `/' from member names 2026-07-24T07:10:44.630275301Z tar: Removing leading `/' from member names 2026-07-24T07:10:44.661362618Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:44.704019441Z Wrote inspect data to must-gather. 2026-07-24T07:10:44.756434111Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-24T07:10:44.910875986Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:44.915245012Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-24T07:10:44.948827849Z W0724 07:10:44.948779 1422 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in '' 2026-07-24T07:10:44.956389339Z Wrote inspect data to must-gather. 2026-07-24T07:10:44.971814629Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-24T07:10:45.041234237Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-24T07:10:45.141867016Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:45.153581418Z INFO: Waiting for network log collection to complete ... 2026-07-24T07:10:45.153687423Z INFO: Waiting for ovnk database copies to complete ... 2026-07-24T07:10:45.154185433Z INFO: Copying ovnk databases complete. 2026-07-24T07:10:45.156940423Z 2.7M must-gather/network_logs/ovnk_database_store 2026-07-24T07:10:45.159560255Z ovnk_database_store/ 2026-07-24T07:10:45.159614626Z ovnk_database_store/ovnkube-node-77hzx_sbdb 2026-07-24T07:10:45.180400104Z ovnk_database_store/ovnkube-node-xls2w_nbdb 2026-07-24T07:10:45.187608977Z ovnk_database_store/ovnkube-node-rcqc6_sbdb 2026-07-24T07:10:45.197580521Z Gathering data for ns/openshift-network-console... 2026-07-24T07:10:45.204436184Z ovnk_database_store/ovnkube-node-rcqc6_nbdb 2026-07-24T07:10:45.211984054Z ovnk_database_store/ovnkube-node-xls2w_sbdb 2026-07-24T07:10:45.229196698Z ovnk_database_store/ovnkube-node-77hzx_nbdb 2026-07-24T07:10:45.238266742Z INFO: Network log collection complete. 2026-07-24T07:10:45.283010977Z Wrote inspect data to must-gather. 2026-07-24T07:10:45.372071935Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:45.374645797Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-24T07:10:45.474781219Z INFO: Getting status from alertmanager-main-0 2026-07-24T07:10:45.568268262Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-24T07:10:45.572079287Z INFO: Fallback to identify the container image from release info 2026-07-24T07:10:45.594566353Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:45.674193788Z Gathering data for ns/openshift-console-operator... 2026-07-24T07:10:45.803564368Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:45.805682451Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-24T07:10:46.004450330Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:46.037291365Z Gathering data for ns/openshift-console... 2026-07-24T07:10:46.088570790Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811 2026-07-24T07:10:46.202265159Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:46.205036142Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-24T07:10:46.224191293Z daemonset.apps/perf-node-gather-daemonset created 2026-07-24T07:10:46.353232114Z Waiting for performance profile collector pods to become ready: 1 2026-07-24T07:10:46.373025942Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:46.572297336Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-24T07:10:46.573969244Z Wrote inspect data to /must-gather/istio. 2026-07-24T07:10:47.101104176Z Gathering data for ns/openshift-dns-operator... 2026-07-24T07:10:47.245670079Z Done executing Istio gather script 2026-07-24T07:10:47.370482615Z Gathering data for ns/openshift-dns... 2026-07-24T07:10:47.461362990Z Waiting for performance profile collector pods to become ready: 2 2026-07-24T07:10:47.887496681Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-07-24T07:10:47.906112605Z Gathering data for ns/openshift-image-registry... 2026-07-24T07:10:48.316636239Z Gathering data for ns/openshift-ingress-operator... 2026-07-24T07:10:48.586069797Z Waiting for performance profile collector pods to become ready: 3 2026-07-24T07:10:48.613199720Z Gathering data for ns/openshift-ingress... 2026-07-24T07:10:48.904380877Z Gathering data for ns/openshift-ingress-canary... 2026-07-24T07:10:49.250838010Z Gathering data for ns/openshift-insights... 2026-07-24T07:10:49.703876522Z Waiting for performance profile collector pods to become ready: 4 2026-07-24T07:10:50.811524235Z Waiting for performance profile collector pods to become ready: 5 2026-07-24T07:10:50.978816200Z Gathering data for ns/openshift-jobset-operator... 2026-07-24T07:10:51.724952510Z Gathering data for ns/openshift-config... 2026-07-24T07:10:51.938833922Z Waiting for performance profile collector pods to become ready: 6 2026-07-24T07:10:51.970957504Z Gathering data for ns/openshift-config-managed... 2026-07-24T07:10:52.358128667Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-24T07:10:52.595756537Z Gathering data for ns/openshift-kube-apiserver... 2026-07-24T07:10:52.826601474Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-24T07:10:52.892447550Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-07-24T07:10:53.061549728Z Waiting for performance profile collector pods to become ready: 7 2026-07-24T07:10:53.099209821Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-24T07:10:53.357923687Z Gathering data for ns/openshift-kube-scheduler... 2026-07-24T07:10:53.584590303Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-24T07:10:53.842472349Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-24T07:10:54.132028329Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-24T07:10:54.188016846Z Waiting for performance profile collector pods to become ready: 8 2026-07-24T07:10:54.418234916Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-24T07:10:54.859420893Z Gathering data for ns/openshift-multus... 2026-07-24T07:10:55.289972222Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-24T07:10:55.506491754Z Collecting performance related data for node ip-10-0-128-250.ec2.internal 2026-07-24T07:10:55.514067118Z Collecting performance related data for node ip-10-0-138-219.ec2.internal 2026-07-24T07:10:55.521723766Z Collecting performance related data for node ip-10-0-135-109.ec2.internal 2026-07-24T07:10:56.317056473Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-24T07:10:57.479793027Z Gathering data for ns/openshift-host-network... 2026-07-24T07:10:57.775611347Z Gathering data for ns/openshift-network-diagnostics... 2026-07-24T07:10:57.897729223Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-07-24T07:10:58.316103263Z Gathering data for ns/openshift-network-node-identity... 2026-07-24T07:10:58.733754914Z Gathering data for ns/openshift-network-operator... 2026-07-24T07:10:59.087119879Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-24T07:10:59.324930281Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-24T07:10:59.784580305Z Gathering data for ns/openshift-apiserver-operator... 2026-07-24T07:11:00.062174015Z Gathering data for ns/openshift-apiserver... 2026-07-24T07:11:00.437323738Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-24T07:11:00.678993358Z Gathering data for ns/openshift-controller-manager... 2026-07-24T07:11:00.944458914Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-24T07:11:01.571442808Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-24T07:11:01.840057882Z Gathering data for ns/openshift-service-ca-operator... 2026-07-24T07:11:02.134226689Z Gathering data for ns/openshift-service-ca... 2026-07-24T07:11:02.466863492Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-24T07:11:02.903068360Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70 2026-07-24T07:11:03.348374467Z Wrote inspect data to must-gather. 2026-07-24T07:11:03.348413890Z error: inspection completed with the errors occurred while gathering data: 2026-07-24T07:11:03.348413890Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-24T07:11:03.736969266Z Collecting kubelet logs for node ip-10-0-128-250.ec2.internal 2026-07-24T07:11:03.738773500Z Collecting kubelet logs for node ip-10-0-138-219.ec2.internal 2026-07-24T07:11:03.740505867Z Collecting kubelet logs for node ip-10-0-135-109.ec2.internal 2026-07-24T07:11:04.035819713Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-lpqmk namespace 2026-07-24T07:11:04.041350651Z INFO: Node performance data collection complete. 2026-07-24T07:11:04.170054696Z error: the server doesn't have a resource type "clusters" 2026-07-24T07:11:04.325638423Z Caches written to disk