2026-06-03T22:08:39.331896879Z [disk usage checker] Started
2026-06-03T22:08:39.335545356Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-03T22:08:39.528297594Z Gathering data for ns/openshift-cluster-version...
2026-06-03T22:08:39.636565860Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T22:08:39.724020524Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T22:08:39.799743687Z Gathering data for ns/default...
2026-06-03T22:08:40.091059681Z Gathering data for ns/openshift...
2026-06-03T22:08:40.499881962Z Gathering data for ns/kube-system...
2026-06-03T22:08:41.182364934Z Gathering data for ns/openshift-etcd...
2026-06-03T22:08:41.477870743Z Wrote inspect data to must-gather.
2026-06-03T22:08:41.477905417Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T22:08:41.477905417Z     namespaces "assisted-installer" not found
2026-06-03T22:08:41.878724641Z Waiting on subprocesses to finish execution.
2026-06-03T22:08:41.897764969Z Executing Istio gather script
2026-06-03T22:08:41.901333360Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-03T22:08:41.905714934Z INFO: Gathering HAProxy config files
2026-06-03T22:08:41.908066025Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-03T22:08:41.918823208Z INFO: Collecting host service logs for crio
2026-06-03T22:08:41.919360845Z INFO: Collecting host service logs for kubelet
2026-06-03T22:08:41.919793563Z INFO: Collecting host service logs for rpm-ostreed
2026-06-03T22:08:41.920151018Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-03T22:08:41.920502799Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-03T22:08:41.920854745Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-03T22:08:41.921227476Z INFO: Collecting host service logs for NetworkManager
2026-06-03T22:08:41.921586762Z INFO: Collecting host service logs for openvswitch
2026-06-03T22:08:41.921920654Z INFO: Collecting host service logs for ovs-configuration
2026-06-03T22:08:41.922271594Z INFO: Collecting host service logs for ovsdb-server
2026-06-03T22:08:41.922621508Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-03T22:08:41.923043861Z INFO: Waiting for worker host service log collection to complete ...
2026-06-03T22:08:41.924460970Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-03T22:08:41.951363292Z INFO: Waiting for node performance related collection to complete ...
2026-06-03T22:08:42.309847565Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-03T22:08:42.511470365Z No resources found in openshift-etcd namespace.
2026-06-03T22:08:42.646018623Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-03T22:08:42.667907932Z Wrote inspect data to must-gather.
2026-06-03T22:08:42.721151133Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-03T22:08:42.832316026Z error: the server doesn't have a resource type "performanceprofile"
2026-06-03T22:08:42.843175120Z INFO: Collecting Insights Archives from insights-operator-69684796bc-s7p9w
2026-06-03T22:08:42.843175120Z insights-runtime-extractor-7jxnj
2026-06-03T22:08:42.843175120Z insights-runtime-extractor-cq2fx
2026-06-03T22:08:42.843175120Z insights-runtime-extractor-zzbjs
2026-06-03T22:08:42.926253364Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-03T22:08:43.208206128Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-03T22:08:43.248675341Z ERROR: No running kube-apiserver pods found
2026-06-03T22:08:43.251380224Z No resources found
2026-06-03T22:08:43.345931020Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-03T22:08:43.345931020Z See 'oc rsync -h' for help and examples
2026-06-03T22:08:43.355201288Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-03T22:08:43.355201288Z INFO: on-disk MachineConfig config collection complete.
2026-06-03T22:08:43.400832748Z INFO: "metallb-operator" not detected. Skipping.
2026-06-03T22:08:43.554315673Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-03T22:08:43.734638955Z INFO: Worker host service log collection to complete.
2026-06-03T22:08:43.736089848Z Wrote inspect data to must-gather.
2026-06-03T22:08:43.905084910Z Wrote inspect data to must-gather.
2026-06-03T22:08:43.949405088Z Wrote inspect data to must-gather.
2026-06-03T22:08:44.077177964Z tar: Removing leading `/' from member names
2026-06-03T22:08:44.093143890Z INFO: HAProxy config collection complete.
2026-06-03T22:08:44.219226172Z INFO: 'previous-logs' folder not found on ip-10-0-134-237.ec2.internal, skipping...
2026-06-03T22:08:44.221067808Z No resources found
2026-06-03T22:08:44.232420387Z Wrote inspect data to must-gather.
2026-06-03T22:08:44.235921974Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T22:08:44.238649608Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-03T22:08:44.371566596Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-03T22:08:44.373400472Z Wrote inspect data to must-gather.
2026-06-03T22:08:44.390696598Z Wrote inspect data to must-gather.
2026-06-03T22:08:44.440885863Z error: the server doesn't have a resource type "machineconfigs"
2026-06-03T22:08:44.445834954Z INFO: Getting rules from prometheus-k8s-0
2026-06-03T22:08:44.603297109Z Gathering data for ns/openshift-monitoring...
2026-06-03T22:08:44.611424417Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-03T22:08:44.626751671Z INFO: 'previous-logs' folder not found on ip-10-0-138-139.ec2.internal, skipping...
2026-06-03T22:08:44.654104601Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-03T22:08:44.696491436Z Wrote inspect data to must-gather.
2026-06-03T22:08:44.873510418Z Wrote inspect data to must-gather.
2026-06-03T22:08:44.900281412Z INFO: 'previous-logs' folder not found on ip-10-0-143-58.ec2.internal, skipping...
2026-06-03T22:08:44.900316524Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-03T22:08:44.900343525Z INFO: Machine Config Daemon termination log collection complete.
2026-06-03T22:08:44.915808446Z INFO: Getting status/config from prometheus-k8s-0
2026-06-03T22:08:44.953145114Z Wrote inspect data to must-gather.
2026-06-03T22:08:45.031279582Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-03T22:08:45.196589854Z Wrote inspect data to must-gather.
2026-06-03T22:08:45.223938372Z Wrote inspect data to must-gather.
2026-06-03T22:08:45.369194661Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-03T22:08:45.411803619Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-03T22:08:45.451004499Z Wrote inspect data to must-gather.
2026-06-03T22:08:45.472519112Z INFO: INTERCONNECT MODE
2026-06-03T22:08:45.472555675Z INFO: Gathering ovn-kubernetes DBs
2026-06-03T22:08:45.594381027Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:45.630520235Z INFO: Gathering OVN_Northbound from ovnkube-node-46r9m...
2026-06-03T22:08:45.630724423Z INFO: Gathering OVN_Northbound from ovnkube-node-m7g24...
2026-06-03T22:08:45.630923225Z INFO: Gathering OVN_Northbound from ovnkube-node-sh4x6...
2026-06-03T22:08:45.631191732Z INFO: Gathering OVN_Southbound from ovnkube-node-46r9m...
2026-06-03T22:08:45.631442152Z INFO: Gathering OVN_Southbound from ovnkube-node-m7g24...
2026-06-03T22:08:45.631693929Z INFO: Gathering OVN_Southbound from ovnkube-node-sh4x6...
2026-06-03T22:08:45.661068081Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-03T22:08:45.667887000Z Wrote inspect data to must-gather.
2026-06-03T22:08:45.915635281Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:45.918348867Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-03T22:08:45.939265835Z tar: Removing leading `/' from member names
2026-06-03T22:08:45.959450425Z tar: Removing leading `/' from member names
2026-06-03T22:08:45.964926348Z tar: Removing leading `/' from member names
2026-06-03T22:08:45.972772651Z tar: Removing leading `/' from member names
2026-06-03T22:08:45.994142256Z tar: Removing leading `/' from member names
2026-06-03T22:08:46.010605552Z tar: Removing leading `/' from member names
2026-06-03T22:08:46.015697173Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-03T22:08:46.047491410Z Wrote inspect data to must-gather.
2026-06-03T22:08:46.132337652Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:46.283936831Z Wrote inspect data to must-gather.
2026-06-03T22:08:46.315951955Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-03T22:08:46.381648247Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:46.385877044Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-03T22:08:46.461230819Z INFO: Waiting for network log collection to complete ...
2026-06-03T22:08:46.461283664Z INFO: Waiting for ovnk database copies to complete ...
2026-06-03T22:08:46.461773058Z INFO: Copying ovnk databases complete.
2026-06-03T22:08:46.463595871Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-06-03T22:08:46.466212055Z ovnk_database_store/
2026-06-03T22:08:46.466212055Z ovnk_database_store/ovnkube-node-sh4x6_nbdb
2026-06-03T22:08:46.473928897Z ovnk_database_store/ovnkube-node-sh4x6_sbdb
2026-06-03T22:08:46.491635243Z ovnk_database_store/ovnkube-node-46r9m_nbdb
2026-06-03T22:08:46.498454596Z ovnk_database_store/ovnkube-node-m7g24_sbdb
2026-06-03T22:08:46.514694792Z ovnk_database_store/ovnkube-node-m7g24_nbdb
2026-06-03T22:08:46.519880967Z Wrote inspect data to must-gather.
2026-06-03T22:08:46.520967781Z ovnk_database_store/ovnkube-node-46r9m_sbdb
2026-06-03T22:08:46.522315274Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-03T22:08:46.537432750Z INFO: Network log collection complete.
2026-06-03T22:08:46.590896474Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:46.714311902Z Wrote inspect data to must-gather.
2026-06-03T22:08:46.762484183Z INFO: Getting status from alertmanager-main-0
2026-06-03T22:08:46.806848078Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:46.810239944Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-03T22:08:46.982820685Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-03T22:08:46.985614162Z INFO: Fallback to identify the container image from release info
2026-06-03T22:08:47.034938757Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:47.158669367Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-03T22:08:47.232168157Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:47.234969381Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-03T22:08:47.333763511Z Gathering data for ns/openshift-network-console...
2026-06-03T22:08:47.383621968Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:47.586256661Z Wrote inspect data to /must-gather/istio.
2026-06-03T22:08:47.662623796Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6566c94ad72a237c5dec66f76fcbc92b95df90f391fc7a044bce3efe63d698e2
2026-06-03T22:08:47.800755591Z daemonset.apps/perf-node-gather-daemonset created
2026-06-03T22:08:47.913689593Z Gathering data for ns/openshift-console-operator...
2026-06-03T22:08:47.937424858Z Waiting for performance profile collector pods to become ready: 1
2026-06-03T22:08:48.281543961Z Done executing Istio gather script
2026-06-03T22:08:48.306336884Z Gathering data for ns/openshift-console...
2026-06-03T22:08:48.771262748Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-03T22:08:49.058064751Z Waiting for performance profile collector pods to become ready: 2
2026-06-03T22:08:49.245681768Z Gathering data for ns/openshift-dns-operator...
2026-06-03T22:08:49.385367194Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-03T22:08:49.511421500Z Gathering data for ns/openshift-dns...
2026-06-03T22:08:50.137795602Z Gathering data for ns/openshift-image-registry...
2026-06-03T22:08:50.170745271Z Waiting for performance profile collector pods to become ready: 3
2026-06-03T22:08:50.584351587Z Gathering data for ns/openshift-ingress-operator...
2026-06-03T22:08:50.884215748Z Gathering data for ns/openshift-ingress...
2026-06-03T22:08:51.189391356Z Gathering data for ns/openshift-ingress-canary...
2026-06-03T22:08:51.270783045Z Waiting for performance profile collector pods to become ready: 4
2026-06-03T22:08:51.563021223Z Gathering data for ns/openshift-insights...
2026-06-03T22:08:52.375618877Z Waiting for performance profile collector pods to become ready: 5
2026-06-03T22:08:53.331337264Z Gathering data for ns/openshift-config...
2026-06-03T22:08:53.484024554Z Waiting for performance profile collector pods to become ready: 6
2026-06-03T22:08:53.599297255Z Gathering data for ns/openshift-config-managed...
2026-06-03T22:08:53.986482767Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-03T22:08:54.223971350Z Gathering data for ns/openshift-kube-apiserver...
2026-06-03T22:08:54.391752461Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-03T22:08:54.466878419Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-03T22:08:54.596708993Z Waiting for performance profile collector pods to become ready: 7
2026-06-03T22:08:54.731860706Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-03T22:08:55.015802046Z Gathering data for ns/openshift-kube-scheduler...
2026-06-03T22:08:55.307249615Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-03T22:08:55.623080729Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-03T22:08:55.704973482Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-03T22:08:55.945075469Z Collecting performance related data for node ip-10-0-143-58.ec2.internal
2026-06-03T22:08:55.945949572Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-03T22:08:55.951913081Z Collecting performance related data for node ip-10-0-138-139.ec2.internal
2026-06-03T22:08:55.958971572Z Collecting performance related data for node ip-10-0-134-237.ec2.internal
2026-06-03T22:08:56.318923434Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-03T22:08:56.889836393Z Gathering data for ns/openshift-multus...
2026-06-03T22:08:58.381228243Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-03T22:08:59.398921495Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-03T22:08:59.588322213Z Gathering data for ns/openshift-host-network...
2026-06-03T22:08:59.881263973Z Gathering data for ns/openshift-network-diagnostics...
2026-06-03T22:09:00.414975307Z Gathering data for ns/openshift-network-node-identity...
2026-06-03T22:09:00.833503092Z Gathering data for ns/openshift-network-operator...
2026-06-03T22:09:01.243307354Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-03T22:09:01.574285237Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-03T22:09:02.058884391Z Gathering data for ns/openshift-apiserver-operator...
2026-06-03T22:09:02.344485997Z Gathering data for ns/openshift-apiserver...
2026-06-03T22:09:02.725553848Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-03T22:09:02.991459079Z Gathering data for ns/openshift-controller-manager...
2026-06-03T22:09:03.255362146Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-03T22:09:03.658856687Z Collecting kubelet logs for node ip-10-0-143-58.ec2.internal
2026-06-03T22:09:03.660595562Z Collecting kubelet logs for node ip-10-0-138-139.ec2.internal
2026-06-03T22:09:03.662348328Z Collecting kubelet logs for node ip-10-0-134-237.ec2.internal
2026-06-03T22:09:03.928807641Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-03T22:09:03.986298877Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-hnj5l namespace
2026-06-03T22:09:03.993033755Z INFO: Node performance data collection complete.
2026-06-03T22:09:04.226372307Z Gathering data for ns/openshift-service-ca-operator...
2026-06-03T22:09:04.405295915Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-03T22:09:04.539175506Z Gathering data for ns/openshift-service-ca...
2026-06-03T22:09:04.867161885Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-03T22:09:05.736803789Z Wrote inspect data to must-gather.
2026-06-03T22:09:05.736844858Z error: inspection completed with the errors occurred while gathering data:
2026-06-03T22:09:05.736844858Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-03T22:09:05.857836450Z error: the server doesn't have a resource type "clusters"
2026-06-03T22:09:05.984977473Z Caches written to disk