2026-07-02T13:06:30.509856310Z [disk usage checker] Started 2026-07-02T13:06:30.513381372Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:06:30.717310728Z Gathering data for ns/openshift-cluster-version... 2026-07-02T13:06:30.864933207Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-02T13:06:30.983177690Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-02T13:06:31.126951249Z Gathering data for ns/default... 2026-07-02T13:06:31.497832718Z Gathering data for ns/openshift... 2026-07-02T13:06:31.983361661Z Gathering data for ns/kube-system... 2026-07-02T13:06:32.972116148Z Gathering data for ns/openshift-etcd... 2026-07-02T13:06:33.388732999Z Wrote inspect data to must-gather. 2026-07-02T13:06:33.388764103Z error: inspection completed with the errors occurred while gathering data: 2026-07-02T13:06:33.388764103Z namespaces "assisted-installer" not found 2026-07-02T13:06:33.397078484Z Waiting on subprocesses to finish execution. 2026-07-02T13:06:33.402457023Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-07-02T13:06:33.408414937Z INFO: Gathering machine config daemon's old logs from all nodes 2026-07-02T13:06:33.408976278Z INFO: Collecting host service logs for crio 2026-07-02T13:06:33.409330916Z INFO: Collecting host service logs for kubelet 2026-07-02T13:06:33.409763996Z INFO: Collecting host service logs for rpm-ostreed 2026-07-02T13:06:33.410115747Z INFO: Collecting host service logs for ostree-finalize-staged 2026-07-02T13:06:33.410493002Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-07-02T13:06:33.410904906Z INFO: Collecting host service logs for machine-config-daemon-host 2026-07-02T13:06:33.411292331Z INFO: Collecting host service logs for NetworkManager 2026-07-02T13:06:33.411965695Z INFO: Collecting host service logs for openvswitch 2026-07-02T13:06:33.412271470Z INFO: Collecting host service logs for ovs-configuration 2026-07-02T13:06:33.412679965Z INFO: Collecting host service logs for ovsdb-server 2026-07-02T13:06:33.412982838Z INFO: Collecting host service logs for ovs-vswitchd 2026-07-02T13:06:33.413379027Z INFO: Waiting for worker host service log collection to complete ... 2026-07-02T13:06:33.415474971Z Executing Istio gather script 2026-07-02T13:06:33.415711025Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-07-02T13:06:33.426552989Z INFO: Gathering HAProxy config files 2026-07-02T13:06:33.537493138Z INFO: Waiting for node performance related collection to complete ... 2026-07-02T13:06:33.863994390Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-07-02T13:06:34.019754141Z INFO: Found 1 replicas - prometheus-k8s-0 2026-07-02T13:06:34.142982905Z Wrote inspect data to must-gather. 2026-07-02T13:06:34.360887849Z INFO: "metallb-operator" not detected. Skipping. 2026-07-02T13:06:34.386701233Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-07-02T13:06:34.386741254Z INFO: on-disk MachineConfig config collection complete. 2026-07-02T13:06:34.445411073Z No resources found in openshift-etcd namespace. 2026-07-02T13:06:34.486486885Z ERROR: No running kube-apiserver pods found 2026-07-02T13:06:34.715209178Z error: the server doesn't have a resource type "performanceprofile" 2026-07-02T13:06:34.739076415Z INFO: "sriov-network-operator" not detected. Skipping. 2026-07-02T13:06:34.818572403Z No resources found 2026-07-02T13:06:34.844659610Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-07-02T13:06:34.849257839Z INFO: Collecting Insights Archives from insights-operator-5f67b55555-v24km 2026-07-02T13:06:34.849257839Z insights-runtime-extractor-8s8v2 2026-07-02T13:06:34.849257839Z insights-runtime-extractor-ksn6z 2026-07-02T13:06:34.849257839Z insights-runtime-extractor-r2zs8 2026-07-02T13:06:34.916755187Z INFO: Worker host service log collection to complete. 2026-07-02T13:06:35.192783873Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-07-02T13:06:35.309364465Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-02T13:06:35.385216976Z INFO: 'previous-logs' folder not found on ip-10-0-134-129.ec2.internal, skipping... 2026-07-02T13:06:35.574060717Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-07-02T13:06:35.596976499Z Wrote inspect data to must-gather. 2026-07-02T13:06:35.603822273Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-07-02T13:06:35.603822273Z See 'oc rsync -h' for help and examples 2026-07-02T13:06:35.606175913Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:06:35.610722993Z Wrote inspect data to must-gather. 2026-07-02T13:06:35.800771135Z Wrote inspect data to must-gather. 2026-07-02T13:06:35.810543418Z Gathering data for ns/openshift-monitoring... 2026-07-02T13:06:35.892258985Z Wrote inspect data to must-gather. 2026-07-02T13:06:35.986008728Z INFO: Waiting for HAProxy config collection to complete ... 2026-07-02T13:06:36.021449903Z No resources found 2026-07-02T13:06:36.044948787Z Wrote inspect data to must-gather. 2026-07-02T13:06:36.068549000Z Wrote inspect data to must-gather. 2026-07-02T13:06:36.228380596Z error: the server doesn't have a resource type "machineconfigs" 2026-07-02T13:06:36.232558335Z INFO: 'previous-logs' folder not found on ip-10-0-137-242.ec2.internal, skipping... 2026-07-02T13:06:36.272261377Z Wrote inspect data to must-gather. 2026-07-02T13:06:36.407241128Z tar: Removing leading `/' from member names 2026-07-02T13:06:36.415529916Z INFO: HAProxy config collection complete. 2026-07-02T13:06:36.428807004Z INFO: Getting rules from prometheus-k8s-0 2026-07-02T13:06:36.446891611Z error: the server doesn't have a resource type "machineconfigpools" 2026-07-02T13:06:36.528725766Z INFO: 'previous-logs' folder not found on ip-10-0-143-149.ec2.internal, skipping... 2026-07-02T13:06:36.528765612Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-07-02T13:06:36.528776040Z INFO: Machine Config Daemon termination log collection complete. 2026-07-02T13:06:36.615374187Z Wrote inspect data to must-gather. 2026-07-02T13:06:36.721056450Z Wrote inspect data to must-gather. 2026-07-02T13:06:36.909889770Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-07-02T13:06:36.915946584Z error: the server doesn't have a resource type "kubeletconfigs" 2026-07-02T13:06:36.934702700Z Wrote inspect data to must-gather. 2026-07-02T13:06:37.077027770Z INFO: Getting status/config from prometheus-k8s-0 2026-07-02T13:06:37.151116453Z Wrote inspect data to must-gather. 2026-07-02T13:06:37.236586574Z INFO: INTERCONNECT MODE 2026-07-02T13:06:37.236628193Z INFO: Gathering ovn-kubernetes DBs 2026-07-02T13:06:37.383341920Z Wrote inspect data to must-gather. 2026-07-02T13:06:37.402429239Z INFO: Gathering OVN_Northbound from ovnkube-node-5vlwp... 2026-07-02T13:06:37.402643042Z INFO: Gathering OVN_Northbound from ovnkube-node-qmx4p... 2026-07-02T13:06:37.402848431Z INFO: Gathering OVN_Northbound from ovnkube-node-w4qr9... 2026-07-02T13:06:37.403115027Z INFO: Gathering OVN_Southbound from ovnkube-node-5vlwp... 2026-07-02T13:06:37.403356042Z INFO: Gathering OVN_Southbound from ovnkube-node-qmx4p... 2026-07-02T13:06:37.403670931Z INFO: Gathering OVN_Southbound from ovnkube-node-w4qr9... 2026-07-02T13:06:37.690936088Z INFO: Getting status/flags from prometheus-k8s-0 2026-07-02T13:06:37.720884634Z Wrote inspect data to must-gather. 2026-07-02T13:06:37.913557141Z tar: Removing leading `/' from member names 2026-07-02T13:06:37.975574729Z tar: Removing leading `/' from member names 2026-07-02T13:06:37.981199149Z Wrote inspect data to must-gather. 2026-07-02T13:06:38.008363742Z tar: Removing leading `/' from member names 2026-07-02T13:06:38.035905825Z tar: Removing leading `/' from member names 2026-07-02T13:06:38.049702431Z tar: Removing leading `/' from member names 2026-07-02T13:06:38.056624392Z tar: Removing leading `/' from member names 2026-07-02T13:06:38.173264604Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-07-02T13:06:38.241133453Z Wrote inspect data to must-gather. 2026-07-02T13:06:38.347010651Z INFO: Waiting for network log collection to complete ... 2026-07-02T13:06:38.347116884Z INFO: Waiting for ovnk database copies to complete ... 2026-07-02T13:06:38.347636356Z INFO: Copying ovnk databases complete. 2026-07-02T13:06:38.349731679Z 3.3M must-gather/network_logs/ovnk_database_store 2026-07-02T13:06:38.352271583Z ovnk_database_store/ 2026-07-02T13:06:38.352271583Z ovnk_database_store/ovnkube-node-5vlwp_sbdb 2026-07-02T13:06:38.371132673Z ovnk_database_store/ovnkube-node-w4qr9_nbdb 2026-07-02T13:06:38.379907550Z ovnk_database_store/ovnkube-node-qmx4p_sbdb 2026-07-02T13:06:38.401129767Z ovnk_database_store/ovnkube-node-w4qr9_sbdb 2026-07-02T13:06:38.417572369Z ovnk_database_store/ovnkube-node-5vlwp_nbdb 2026-07-02T13:06:38.425152827Z ovnk_database_store/ovnkube-node-qmx4p_nbdb 2026-07-02T13:06:38.437223239Z INFO: Network log collection complete. 2026-07-02T13:06:38.458156962Z Wrote inspect data to must-gather. 2026-07-02T13:06:38.459668819Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-07-02T13:06:38.538286173Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-07-02T13:06:38.666016434Z Wrote inspect data to must-gather. 2026-07-02T13:06:38.728146642Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:38.762731531Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-07-02T13:06:38.935302093Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found 2026-07-02T13:06:38.937226132Z INFO: Fallback to identify the container image from release info 2026-07-02T13:06:38.952764736Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:38.955867970Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-07-02T13:06:39.143896206Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:39.187966778Z INFO: Getting status from alertmanager-main-0 2026-07-02T13:06:39.374141965Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:39.376426954Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-07-02T13:06:39.399982029Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+ 2026-07-02T13:06:39.548629170Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2c4f9949a07a5338dd10a5abcb8d6d3f5eebe93333139869f9eec8802800162f 2026-07-02T13:06:39.577196083Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:39.714781357Z daemonset.apps/perf-node-gather-daemonset created 2026-07-02T13:06:39.813667061Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:39.816044323Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-07-02T13:06:39.859882439Z Waiting for performance profile collector pods to become ready: 1 2026-07-02T13:06:40.014586553Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-07-02T13:06:40.063498535Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:40.207814602Z Gathering data for ns/openshift-network-console... 2026-07-02T13:06:40.316959250Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:40.321678394Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io 2026-07-02T13:06:40.506389354Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:40.617255764Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:06:40.724556275Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:40.728204371Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io 2026-07-02T13:06:40.751319227Z Gathering data for ns/openshift-console-operator... 2026-07-02T13:06:40.919041682Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:40.992464848Z Waiting for performance profile collector pods to become ready: 2 2026-07-02T13:06:41.123752311Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:41.126999485Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io 2026-07-02T13:06:41.311296350Z Gathering data for ns/openshift-console... 2026-07-02T13:06:41.332662414Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:41.531705728Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:41.534139613Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-07-02T13:06:41.710863036Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:41.923085754Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:41.925799170Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-07-02T13:06:42.006217702Z Gathering data for ns/openshift-cluster-storage-operator... 2026-07-02T13:06:42.128047553Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:42.135560935Z Waiting for performance profile collector pods to become ready: 3 2026-07-02T13:06:42.318704355Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:42.322067938Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-07-02T13:06:42.510672985Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:42.704237657Z Gathering data for ns/openshift-dns-operator... 2026-07-02T13:06:42.731404450Z Wrote inspect data to /must-gather/istio. 2026-07-02T13:06:43.091579874Z Gathering data for ns/openshift-dns... 2026-07-02T13:06:43.272225120Z Waiting for performance profile collector pods to become ready: 4 2026-07-02T13:06:43.461811199Z Done executing Istio gather script 2026-07-02T13:06:44.029941360Z Gathering data for ns/openshift-image-registry... 2026-07-02T13:06:44.391067901Z Waiting for performance profile collector pods to become ready: 5 2026-07-02T13:06:44.626137630Z Gathering data for ns/openshift-ingress-operator... 2026-07-02T13:06:45.043626037Z Gathering data for ns/openshift-ingress... 2026-07-02T13:06:45.528174349Z Waiting for performance profile collector pods to become ready: 6 2026-07-02T13:06:45.577551255Z Gathering data for ns/openshift-ingress-canary... 2026-07-02T13:06:45.621948277Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:06:46.194701252Z Gathering data for ns/openshift-insights... 2026-07-02T13:06:46.652693700Z Waiting for performance profile collector pods to become ready: 7 2026-07-02T13:06:47.790009776Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-07-02T13:06:48.055428121Z Collecting performance related data for node ip-10-0-134-129.ec2.internal 2026-07-02T13:06:48.062716577Z Collecting performance related data for node ip-10-0-143-149.ec2.internal 2026-07-02T13:06:48.070608009Z Collecting performance related data for node ip-10-0-137-242.ec2.internal 2026-07-02T13:06:49.098920994Z Gathering data for ns/kserve... 2026-07-02T13:06:50.626406165Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:06:51.235563535Z Gathering data for ns/openshift-config... 2026-07-02T13:06:51.681603330Z Gathering data for ns/openshift-config-managed... 2026-07-02T13:06:52.366052271Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-07-02T13:06:52.731334248Z Gathering data for ns/openshift-kube-apiserver... 2026-07-02T13:06:53.077469252Z Gathering data for ns/openshift-kube-controller-manager... 2026-07-02T13:06:53.447002729Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-07-02T13:06:53.832623153Z Gathering data for ns/openshift-kube-scheduler... 2026-07-02T13:06:54.251900712Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-07-02T13:06:54.684275737Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-07-02T13:06:55.176842624Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-07-02T13:06:55.632812125Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:06:55.735931463Z Gathering data for ns/openshift-user-workload-monitoring... 2026-07-02T13:06:56.456490545Z Gathering data for ns/openshift-multus... 2026-07-02T13:06:56.585700585Z Collecting kubelet logs for node ip-10-0-134-129.ec2.internal 2026-07-02T13:06:56.587542316Z Collecting kubelet logs for node ip-10-0-143-149.ec2.internal 2026-07-02T13:06:56.589201561Z Collecting kubelet logs for node ip-10-0-137-242.ec2.internal 2026-07-02T13:06:56.928091093Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-42fsq namespace 2026-07-02T13:06:56.935481064Z INFO: Node performance data collection complete. 2026-07-02T13:06:58.302114812Z Gathering data for ns/openshift-ovn-kubernetes... 2026-07-02T13:07:00.315787214Z Gathering data for ns/openshift-host-network... 2026-07-02T13:07:00.639746740Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:07:00.706357505Z Gathering data for ns/openshift-network-diagnostics... 2026-07-02T13:07:01.389534622Z Gathering data for ns/openshift-network-node-identity... 2026-07-02T13:07:01.884194933Z Gathering data for ns/openshift-network-operator... 2026-07-02T13:07:02.439418707Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-07-02T13:07:02.811034509Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-07-02T13:07:03.417824193Z Gathering data for ns/openshift-apiserver-operator... 2026-07-02T13:07:03.800323514Z Gathering data for ns/openshift-apiserver... 2026-07-02T13:07:04.268811127Z Gathering data for ns/openshift-controller-manager-operator... 2026-07-02T13:07:04.565976483Z Gathering data for ns/openshift-controller-manager... 2026-07-02T13:07:04.867174623Z Gathering data for ns/openshift-cluster-samples-operator... 2026-07-02T13:07:05.542533405Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-07-02T13:07:05.644035185Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70 2026-07-02T13:07:05.894712725Z Gathering data for ns/openshift-service-ca-operator... 2026-07-02T13:07:06.279396239Z Gathering data for ns/openshift-service-ca... 2026-07-02T13:07:06.735098372Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-07-02T13:07:07.821052448Z Wrote inspect data to must-gather. 2026-07-02T13:07:07.821101369Z error: inspection completed with the errors occurred while gathering data: 2026-07-02T13:07:07.821101369Z skipping gathering secrets/support due to error: secrets "support" not found 2026-07-02T13:07:07.954822691Z error: the server doesn't have a resource type "clusters" 2026-07-02T13:07:08.101513910Z Caches written to disk