2026-09-08T11:11:24.882313644Z [disk usage checker] Started 2026-09-08T11:11:24.885866369Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-08T11:11:25.065788410Z Gathering data for ns/openshift-cluster-version... 2026-09-08T11:11:25.196893858Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-08T11:11:25.285352348Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-08T11:11:25.362937592Z Gathering data for ns/default... 2026-09-08T11:11:25.645700127Z Gathering data for ns/openshift... 2026-09-08T11:11:26.082525646Z Gathering data for ns/kube-system... 2026-09-08T11:11:26.770489421Z Gathering data for ns/openshift-etcd... 2026-09-08T11:11:27.085692883Z Wrote inspect data to must-gather. 2026-09-08T11:11:27.085722925Z error: inspection completed with the errors occurred while gathering data: 2026-09-08T11:11:27.085722925Z namespaces "assisted-installer" not found 2026-09-08T11:11:27.293861200Z Waiting on subprocesses to finish execution. 2026-09-08T11:11:27.297643106Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-09-08T11:11:27.302198297Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-09-08T11:11:27.311028105Z INFO: Gathering HAProxy config files 2026-09-08T11:11:27.311844175Z Executing Istio gather script 2026-09-08T11:11:27.325321358Z INFO: Gathering machine config daemon's old logs from all nodes 2026-09-08T11:11:27.330255455Z INFO: Waiting for node performance related collection to complete ... 2026-09-08T11:11:27.375091595Z INFO: Collecting host service logs for crio 2026-09-08T11:11:27.375386674Z INFO: Collecting host service logs for kubelet 2026-09-08T11:11:27.375673330Z INFO: Collecting host service logs for rpm-ostreed 2026-09-08T11:11:27.375945378Z INFO: Collecting host service logs for ostree-finalize-staged 2026-09-08T11:11:27.376226263Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-09-08T11:11:27.376485792Z INFO: Collecting host service logs for machine-config-daemon-host 2026-09-08T11:11:27.376754048Z INFO: Collecting host service logs for NetworkManager 2026-09-08T11:11:27.377018757Z INFO: Collecting host service logs for openvswitch 2026-09-08T11:11:27.377308966Z INFO: Collecting host service logs for ovs-configuration 2026-09-08T11:11:27.377570056Z INFO: Collecting host service logs for ovsdb-server 2026-09-08T11:11:27.377846967Z INFO: Collecting host service logs for ovs-vswitchd 2026-09-08T11:11:27.378164735Z INFO: Waiting for worker host service log collection to complete ... 2026-09-08T11:11:27.719923809Z INFO: Collecting Insights Archives from insights-operator-76bcd6ccf5-b4vmw 2026-09-08T11:11:27.719982275Z insights-runtime-extractor-92bt9 2026-09-08T11:11:27.720000960Z insights-runtime-extractor-ns8ts 2026-09-08T11:11:27.720018987Z insights-runtime-extractor-p55hn 2026-09-08T11:11:28.018905545Z INFO: Found 1 replicas - prometheus-k8s-0 2026-09-08T11:11:28.120820942Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-09-08T11:11:28.120908821Z INFO: on-disk MachineConfig config collection complete. 2026-09-08T11:11:28.141289808Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-09-08T11:11:28.170780625Z No resources found 2026-09-08T11:11:28.176045047Z No resources found in openshift-etcd namespace. 2026-09-08T11:11:28.202278524Z INFO: "metallb-operator" not detected. Skipping. 2026-09-08T11:11:28.262213402Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-09-08T11:11:28.512807024Z INFO: "sriov-network-operator" not detected. Skipping. 2026-09-08T11:11:28.544712640Z error: the server doesn't have a resource type "performanceprofile" 2026-09-08T11:11:28.596561928Z Wrote inspect data to must-gather. 2026-09-08T11:11:28.660985294Z Wrote inspect data to must-gather. 2026-09-08T11:11:28.705639836Z INFO: Waiting for HAProxy config collection to complete ... 2026-09-08T11:11:28.748579183Z INFO: Worker host service log collection to complete. 2026-09-08T11:11:28.811278111Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-09-08T11:11:28.811278111Z See 'oc rsync -h' for help and examples 2026-09-08T11:11:28.910765304Z ERROR: No running kube-apiserver pods found 2026-09-08T11:11:29.014471119Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-09-08T11:11:29.147497873Z INFO: 'previous-logs' folder not found on ip-10-0-132-189.ec2.internal, skipping... 2026-09-08T11:11:29.213386243Z Wrote inspect data to must-gather. 2026-09-08T11:11:29.218065411Z Wrote inspect data to must-gather. 2026-09-08T11:11:29.395281641Z Gathering data for ns/openshift-monitoring... 2026-09-08T11:11:29.444347049Z tar: Removing leading `/' from member names 2026-09-08T11:11:29.449882514Z INFO: HAProxy config collection complete. 2026-09-08T11:11:29.566428149Z Wrote inspect data to must-gather. 2026-09-08T11:11:29.585189767Z Wrote inspect data to must-gather. 2026-09-08T11:11:29.612545055Z Wrote inspect data to must-gather. 2026-09-08T11:11:29.649310574Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-08T11:11:29.725982563Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-09-08T11:11:29.742178864Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-09-08T11:11:29.758759908Z No resources found 2026-09-08T11:11:29.760626297Z error: the server doesn't have a resource type "machineconfigs" 2026-09-08T11:11:29.861820839Z INFO: 'previous-logs' folder not found on ip-10-0-139-195.ec2.internal, skipping... 2026-09-08T11:11:29.894514918Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-08T11:11:29.931298578Z error: the server doesn't have a resource type "machineconfigpools" 2026-09-08T11:11:30.011792144Z Wrote inspect data to must-gather. 2026-09-08T11:11:30.039802528Z INFO: Getting rules from prometheus-k8s-0 2026-09-08T11:11:30.147633252Z INFO: 'previous-logs' folder not found on ip-10-0-142-173.ec2.internal, skipping... 2026-09-08T11:11:30.147734424Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-09-08T11:11:30.147790290Z INFO: Machine Config Daemon termination log collection complete. 2026-09-08T11:11:30.181403173Z Wrote inspect data to must-gather. 2026-09-08T11:11:30.267141331Z Wrote inspect data to must-gather. 2026-09-08T11:11:30.332052363Z error: the server doesn't have a resource type "kubeletconfigs" 2026-09-08T11:11:30.457065208Z INFO: Getting status/config from prometheus-k8s-0 2026-09-08T11:11:30.509843233Z Wrote inspect data to must-gather. 2026-09-08T11:11:30.542769127Z Wrote inspect data to must-gather. 2026-09-08T11:11:30.613531978Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-09-08T11:11:30.755349620Z Wrote inspect data to must-gather. 2026-09-08T11:11:30.792707006Z INFO: INTERCONNECT MODE 2026-09-08T11:11:30.792761775Z INFO: Gathering ovn-kubernetes DBs 2026-09-08T11:11:30.808179852Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:30.860226585Z INFO: Getting status/flags from prometheus-k8s-0 2026-09-08T11:11:30.986892316Z INFO: Gathering OVN_Northbound from ovnkube-node-ghwhg... 2026-09-08T11:11:30.987274245Z INFO: Gathering OVN_Northbound from ovnkube-node-lmj6g... 2026-09-08T11:11:30.987559069Z INFO: Gathering OVN_Northbound from ovnkube-node-vsqmn... 2026-09-08T11:11:30.987836700Z INFO: Gathering OVN_Southbound from ovnkube-node-ghwhg... 2026-09-08T11:11:30.989413579Z INFO: Gathering OVN_Southbound from ovnkube-node-lmj6g... 2026-09-08T11:11:30.990292741Z INFO: Gathering OVN_Southbound from ovnkube-node-vsqmn... 2026-09-08T11:11:31.007906390Z Wrote inspect data to must-gather. 2026-09-08T11:11:31.060449081Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:31.063600822Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-09-08T11:11:31.114653433Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-09-08T11:11:31.203789041Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-09-08T11:11:31.253995509Z tar: Removing leading `/' from member names 2026-09-08T11:11:31.289788527Z tar: Removing leading `/' from member names 2026-09-08T11:11:31.324252407Z tar: Removing leading `/' from member names 2026-09-08T11:11:31.332451777Z tar: Removing leading `/' from member names 2026-09-08T11:11:31.335223959Z tar: Removing leading `/' from member names 2026-09-08T11:11:31.356700573Z tar: Removing leading `/' from member names 2026-09-08T11:11:31.369487900Z Wrote inspect data to must-gather. 2026-09-08T11:11:31.386576999Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:31.498015137Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-09-08T11:11:31.630256040Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:31.634086945Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-09-08T11:11:31.634675317Z Wrote inspect data to must-gather. 2026-09-08T11:11:31.766057833Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-09-08T11:11:31.829526188Z INFO: Waiting for network log collection to complete ... 2026-09-08T11:11:31.830048463Z INFO: Waiting for ovnk database copies to complete ... 2026-09-08T11:11:31.831362411Z INFO: Copying ovnk databases complete. 2026-09-08T11:11:31.833969143Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-09-08T11:11:31.834286914Z 2.4M must-gather/network_logs/ovnk_database_store 2026-09-08T11:11:31.837015510Z ovnk_database_store/ 2026-09-08T11:11:31.837046750Z ovnk_database_store/ovnkube-node-lmj6g_nbdb 2026-09-08T11:11:31.845655048Z ovnk_database_store/ovnkube-node-lmj6g_sbdb 2026-09-08T11:11:31.851239880Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:31.860581357Z ovnk_database_store/ovnkube-node-ghwhg_nbdb 2026-09-08T11:11:31.867128623Z ovnk_database_store/ovnkube-node-ghwhg_sbdb 2026-09-08T11:11:31.884514022Z ovnk_database_store/ovnkube-node-vsqmn_sbdb 2026-09-08T11:11:31.902656276Z ovnk_database_store/ovnkube-node-vsqmn_nbdb 2026-09-08T11:11:31.912036162Z INFO: Network log collection complete. 2026-09-08T11:11:31.950619359Z Wrote inspect data to must-gather. 2026-09-08T11:11:32.022838224Z Gathering data for ns/openshift-network-console... 2026-09-08T11:11:32.059870400Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:32.063782580Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-09-08T11:11:32.249778973Z Wrote inspect data to must-gather. 2026-09-08T11:11:32.291797444Z INFO: Getting status from alertmanager-main-0 2026-09-08T11:11:32.322751205Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:32.529264499Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-09-08T11:11:32.531400707Z INFO: Fallback to identify the container image from release info 2026-09-08T11:11:32.541607225Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:32.543835695Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-09-08T11:11:32.552710751Z Gathering data for ns/openshift-console-operator... 2026-09-08T11:11:32.706665951Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:32.871406687Z Wrote inspect data to /must-gather/istio. 2026-09-08T11:11:32.938760798Z Gathering data for ns/openshift-console... 2026-09-08T11:11:33.425901339Z Gathering data for ns/openshift-cluster-storage-operator... 2026-09-08T11:11:33.463791184Z Done executing Istio gather script 2026-09-08T11:11:33.875139573Z Gathering data for ns/openshift-dns-operator... 2026-09-08T11:11:34.146040351Z Gathering data for ns/openshift-dns... 2026-09-08T11:11:34.601425052Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f39c1daa89e19c5e987a3b759e646ffab6c1b2ab94a97ded81ddf13e73d0399 2026-09-08T11:11:34.720674913Z daemonset.apps/perf-node-gather-daemonset created 2026-09-08T11:11:34.821070302Z Gathering data for ns/openshift-image-registry... 2026-09-08T11:11:34.838653969Z Waiting for performance profile collector pods to become ready: 1 2026-09-08T11:11:34.899582676Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-08T11:11:35.295716565Z Gathering data for ns/openshift-ingress-operator... 2026-09-08T11:11:35.602743895Z Gathering data for ns/openshift-ingress... 2026-09-08T11:11:35.910065207Z Gathering data for ns/openshift-ingress-canary... 2026-09-08T11:11:35.949542573Z Waiting for performance profile collector pods to become ready: 2 2026-09-08T11:11:36.267240462Z Gathering data for ns/openshift-insights... 2026-09-08T11:11:37.049418921Z Waiting for performance profile collector pods to become ready: 3 2026-09-08T11:11:38.030094834Z Gathering data for ns/openshift-config... 2026-09-08T11:11:38.159440805Z Waiting for performance profile collector pods to become ready: 4 2026-09-08T11:11:38.286123773Z Gathering data for ns/openshift-config-managed... 2026-09-08T11:11:38.677582668Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-09-08T11:11:38.971655888Z Gathering data for ns/openshift-kube-apiserver... 2026-09-08T11:11:39.229571680Z Gathering data for ns/openshift-kube-controller-manager... 2026-09-08T11:11:39.279061488Z Waiting for performance profile collector pods to become ready: 5 2026-09-08T11:11:39.504711284Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-09-08T11:11:39.785575329Z Gathering data for ns/openshift-kube-scheduler... 2026-09-08T11:11:39.904129957Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-08T11:11:40.043866800Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-09-08T11:11:40.333233495Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-09-08T11:11:40.385929863Z Waiting for performance profile collector pods to become ready: 6 2026-09-08T11:11:40.675098683Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-09-08T11:11:40.966686597Z Gathering data for ns/openshift-user-workload-monitoring... 2026-09-08T11:11:41.426094550Z Gathering data for ns/openshift-multus... 2026-09-08T11:11:41.490012669Z Waiting for performance profile collector pods to become ready: 7 2026-09-08T11:11:42.589701710Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-09-08T11:11:42.828794898Z Collecting performance related data for node ip-10-0-139-195.ec2.internal 2026-09-08T11:11:42.836669501Z Collecting performance related data for node ip-10-0-132-189.ec2.internal 2026-09-08T11:11:42.844803195Z Collecting performance related data for node ip-10-0-142-173.ec2.internal 2026-09-08T11:11:43.062823050Z Gathering data for ns/openshift-ovn-kubernetes... 2026-09-08T11:11:44.456420159Z Gathering data for ns/openshift-host-network... 2026-09-08T11:11:44.768382747Z Gathering data for ns/openshift-network-diagnostics... 2026-09-08T11:11:44.910071547Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-08T11:11:45.379315255Z Gathering data for ns/openshift-network-node-identity... 2026-09-08T11:11:45.780750660Z Gathering data for ns/openshift-network-operator... 2026-09-08T11:11:46.158616652Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-09-08T11:11:46.415189460Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-09-08T11:11:46.916654873Z Gathering data for ns/openshift-apiserver-operator... 2026-09-08T11:11:47.271334691Z Gathering data for ns/openshift-apiserver... 2026-09-08T11:11:47.670719535Z Gathering data for ns/openshift-controller-manager-operator... 2026-09-08T11:11:47.939677534Z Gathering data for ns/openshift-controller-manager... 2026-09-08T11:11:48.219420433Z Gathering data for ns/openshift-cluster-samples-operator... 2026-09-08T11:11:48.844318938Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-09-08T11:11:49.134942831Z Gathering data for ns/openshift-service-ca-operator... 2026-09-08T11:11:49.438395193Z Gathering data for ns/openshift-service-ca... 2026-09-08T11:11:49.820932549Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-09-08T11:11:49.915159711Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70 2026-09-08T11:11:50.707176456Z Collecting kubelet logs for node ip-10-0-139-195.ec2.internal 2026-09-08T11:11:50.709188695Z Collecting kubelet logs for node ip-10-0-132-189.ec2.internal 2026-09-08T11:11:50.711440899Z Collecting kubelet logs for node ip-10-0-142-173.ec2.internal 2026-09-08T11:11:50.749060422Z Wrote inspect data to must-gather. 2026-09-08T11:11:50.749173467Z error: inspection completed with the errors occurred while gathering data: 2026-09-08T11:11:50.749173467Z skipping gathering secrets/support due to error: secrets "support" not found 2026-09-08T11:11:50.994568840Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-b9q8n namespace 2026-09-08T11:11:51.000225247Z INFO: Node performance data collection complete. 2026-09-08T11:11:51.123915677Z error: the server doesn't have a resource type "clusters" 2026-09-08T11:11:51.252871253Z Caches written to disk