2026-04-20T20:28:29.835149657Z [disk usage checker] Started
2026-04-20T20:28:29.869704199Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T20:28:30.059778255Z Gathering data for ns/openshift-cluster-version...
2026-04-20T20:28:30.200556514Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T20:28:30.281981673Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T20:28:30.359283406Z Gathering data for ns/default...
2026-04-20T20:28:30.660914775Z Gathering data for ns/openshift...
2026-04-20T20:28:31.055248996Z Gathering data for ns/kube-system...
2026-04-20T20:28:31.728412452Z Gathering data for ns/openshift-etcd...
2026-04-20T20:28:32.025513866Z Wrote inspect data to must-gather.
2026-04-20T20:28:32.025556586Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T20:28:32.025556586Z     namespaces "assisted-installer" not found
2026-04-20T20:28:32.390504792Z Waiting on subprocesses to finish execution.
2026-04-20T20:28:32.399917136Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-20T20:28:32.403624711Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-20T20:28:32.408587102Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-20T20:28:32.409642487Z INFO: Gathering HAProxy config files
2026-04-20T20:28:32.439229173Z INFO: Collecting host service logs for crio
2026-04-20T20:28:32.439621108Z INFO: Collecting host service logs for kubelet
2026-04-20T20:28:32.439991046Z INFO: Collecting host service logs for rpm-ostreed
2026-04-20T20:28:32.440353086Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-20T20:28:32.440711936Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-20T20:28:32.441044837Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-20T20:28:32.441384251Z INFO: Collecting host service logs for NetworkManager
2026-04-20T20:28:32.441712023Z INFO: Collecting host service logs for openvswitch
2026-04-20T20:28:32.442020004Z INFO: Collecting host service logs for ovs-configuration
2026-04-20T20:28:32.442360520Z INFO: Collecting host service logs for ovsdb-server
2026-04-20T20:28:32.442688196Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-20T20:28:32.443081301Z INFO: Waiting for worker host service log collection to complete ...
2026-04-20T20:28:32.496667264Z INFO: Waiting for node performance related collection to complete ...
2026-04-20T20:28:32.755888849Z No resources found in openshift-etcd namespace.
2026-04-20T20:28:32.818014584Z INFO: "metallb-operator" not detected. Skipping.
2026-04-20T20:28:32.920382195Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-8ftcl
2026-04-20T20:28:32.920443833Z insights-runtime-extractor-58cpq
2026-04-20T20:28:32.920466442Z insights-runtime-extractor-gdfgp
2026-04-20T20:28:32.920486857Z insights-runtime-extractor-q2q25
2026-04-20T20:28:33.240477380Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-20T20:28:33.353958350Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-20T20:28:33.461357145Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-20T20:28:33.461401292Z INFO: on-disk MachineConfig config collection complete.
2026-04-20T20:28:33.536220595Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-20T20:28:33.536220595Z See 'oc rsync -h' for help and examples
2026-04-20T20:28:33.638563851Z error: the server doesn't have a resource type "performanceprofile"
2026-04-20T20:28:33.693632649Z ERROR: No running kube-apiserver pods found
2026-04-20T20:28:33.769888205Z No resources found
2026-04-20T20:28:33.925673594Z Wrote inspect data to must-gather.
2026-04-20T20:28:34.171084792Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-20T20:28:34.225777787Z INFO: Worker host service log collection to complete.
2026-04-20T20:28:34.343545869Z Gathering data for ns/openshift-monitoring...
2026-04-20T20:28:34.344020527Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-20T20:28:34.406995855Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-20T20:28:34.411117493Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T20:28:34.478824303Z No resources found
2026-04-20T20:28:34.567953734Z Wrote inspect data to must-gather.
2026-04-20T20:28:34.590469487Z Wrote inspect data to must-gather.
2026-04-20T20:28:34.663280712Z INFO: 'previous-logs' folder not found on ip-10-0-135-184.ec2.internal, skipping...
2026-04-20T20:28:34.675201071Z Wrote inspect data to must-gather.
2026-04-20T20:28:34.713943461Z Wrote inspect data to must-gather.
2026-04-20T20:28:34.728328795Z Wrote inspect data to must-gather.
2026-04-20T20:28:34.754122893Z Wrote inspect data to must-gather.
2026-04-20T20:28:34.768375418Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-20T20:28:34.826661837Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-20T20:28:34.880819280Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T20:28:34.937676057Z error: the server doesn't have a resource type "machineconfigs"
2026-04-20T20:28:34.954841994Z Wrote inspect data to must-gather.
2026-04-20T20:28:35.010207148Z tar: Removing leading `/' from member names
2026-04-20T20:28:35.016838103Z INFO: HAProxy config collection complete.
2026-04-20T20:28:35.021348869Z INFO: 'previous-logs' folder not found on ip-10-0-136-158.ec2.internal, skipping...
2026-04-20T20:28:35.187835064Z Wrote inspect data to must-gather.
2026-04-20T20:28:35.259589670Z Wrote inspect data to must-gather.
2026-04-20T20:28:35.266671182Z INFO: Getting rules from prometheus-k8s-0
2026-04-20T20:28:35.307653900Z INFO: 'previous-logs' folder not found on ip-10-0-141-130.ec2.internal, skipping...
2026-04-20T20:28:35.307698583Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-20T20:28:35.307730459Z INFO: Machine Config Daemon termination log collection complete.
2026-04-20T20:28:35.431961341Z Wrote inspect data to must-gather.
2026-04-20T20:28:35.515099503Z Wrote inspect data to must-gather.
2026-04-20T20:28:35.595370009Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-20T20:28:35.666226311Z INFO: Getting status/config from prometheus-k8s-0
2026-04-20T20:28:35.844170503Z INFO: INTERCONNECT MODE
2026-04-20T20:28:35.844214033Z INFO: Gathering ovn-kubernetes DBs
2026-04-20T20:28:35.845617292Z Wrote inspect data to must-gather.
2026-04-20T20:28:35.893701510Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T20:28:36.034821644Z INFO: Gathering OVN_Northbound from ovnkube-node-48d2t...
2026-04-20T20:28:36.035169549Z INFO: Gathering OVN_Northbound from ovnkube-node-664zl...
2026-04-20T20:28:36.035485228Z INFO: Gathering OVN_Northbound from ovnkube-node-kldp5...
2026-04-20T20:28:36.035814289Z INFO: Gathering OVN_Southbound from ovnkube-node-48d2t...
2026-04-20T20:28:36.036116012Z INFO: Gathering OVN_Southbound from ovnkube-node-664zl...
2026-04-20T20:28:36.036431568Z INFO: Gathering OVN_Southbound from ovnkube-node-kldp5...
2026-04-20T20:28:36.108630303Z Wrote inspect data to must-gather.
2026-04-20T20:28:36.156419758Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-20T20:28:36.338986773Z tar: Removing leading `/' from member names
2026-04-20T20:28:36.344828347Z tar: Removing leading `/' from member names
2026-04-20T20:28:36.345025929Z tar: Removing leading `/' from member names
2026-04-20T20:28:36.381471232Z tar: Removing leading `/' from member names
2026-04-20T20:28:36.396514798Z tar: Removing leading `/' from member names
2026-04-20T20:28:36.396650750Z tar: Removing leading `/' from member names
2026-04-20T20:28:36.483520982Z Wrote inspect data to must-gather.
2026-04-20T20:28:36.501032412Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-20T20:28:36.632467599Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T20:28:36.777772652Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-20T20:28:36.834176605Z Wrote inspect data to must-gather.
2026-04-20T20:28:36.837623571Z Gathering data for ns/openshift-network-console...
2026-04-20T20:28:36.934272480Z INFO: Waiting for network log collection to complete ...
2026-04-20T20:28:36.934432607Z INFO: Waiting for ovnk database copies to complete ...
2026-04-20T20:28:36.934966336Z INFO: Copying ovnk databases complete.
2026-04-20T20:28:36.938289251Z 3.3M	must-gather/network_logs/ovnk_database_store
2026-04-20T20:28:36.942327984Z ovnk_database_store/
2026-04-20T20:28:36.942384316Z ovnk_database_store/ovnkube-node-kldp5_nbdb
2026-04-20T20:28:36.955400811Z ovnk_database_store/ovnkube-node-48d2t_sbdb
2026-04-20T20:28:36.976041627Z ovnk_database_store/ovnkube-node-48d2t_nbdb
2026-04-20T20:28:36.983288779Z ovnk_database_store/ovnkube-node-kldp5_sbdb
2026-04-20T20:28:37.003932417Z ovnk_database_store/ovnkube-node-664zl_nbdb
2026-04-20T20:28:37.010629051Z ovnk_database_store/ovnkube-node-664zl_sbdb
2026-04-20T20:28:37.034254724Z INFO: Network log collection complete.
2026-04-20T20:28:37.035602276Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-20T20:28:37.180504239Z Wrote inspect data to must-gather.
2026-04-20T20:28:37.297806234Z Gathering data for ns/openshift-console-operator...
2026-04-20T20:28:37.499259960Z Wrote inspect data to must-gather.
2026-04-20T20:28:37.535528038Z INFO: Getting status from alertmanager-main-0
2026-04-20T20:28:37.715614688Z Gathering data for ns/openshift-console...
2026-04-20T20:28:37.808750475Z Wrote inspect data to must-gather.
2026-04-20T20:28:38.079909629Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-20T20:28:38.082875006Z INFO: Fallback to identify the container image from release info
2026-04-20T20:28:38.175316369Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-20T20:28:38.511964249Z Gathering data for ns/openshift-dns-operator...
2026-04-20T20:28:38.771450280Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-20T20:28:38.841438724Z Gathering data for ns/openshift-dns...
2026-04-20T20:28:38.905201345Z daemonset.apps/perf-node-gather-daemonset created
2026-04-20T20:28:39.031030243Z Waiting for performance profile collector pods to become ready: 1
2026-04-20T20:28:39.446751645Z Gathering data for ns/openshift-image-registry...
2026-04-20T20:28:39.884658385Z Gathering data for ns/openshift-ingress-operator...
2026-04-20T20:28:39.886470102Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T20:28:40.176074862Z Waiting for performance profile collector pods to become ready: 2
2026-04-20T20:28:40.208602357Z Gathering data for ns/openshift-ingress...
2026-04-20T20:28:40.516584749Z Gathering data for ns/openshift-ingress-canary...
2026-04-20T20:28:40.879259805Z Gathering data for ns/openshift-insights...
2026-04-20T20:28:41.294867152Z Waiting for performance profile collector pods to become ready: 3
2026-04-20T20:28:42.426043388Z Waiting for performance profile collector pods to become ready: 4
2026-04-20T20:28:42.941734057Z Gathering data for ns/kserve...
2026-04-20T20:28:43.570269206Z Waiting for performance profile collector pods to become ready: 5
2026-04-20T20:28:44.419430580Z Gathering data for ns/openshift-config...
2026-04-20T20:28:44.717575140Z Waiting for performance profile collector pods to become ready: 6
2026-04-20T20:28:44.723923830Z Gathering data for ns/openshift-config-managed...
2026-04-20T20:28:44.891370051Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T20:28:45.128898809Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-20T20:28:45.366512024Z Gathering data for ns/openshift-kube-apiserver...
2026-04-20T20:28:45.610088437Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-20T20:28:45.862334717Z Waiting for performance profile collector pods to become ready: 7
2026-04-20T20:28:45.899554262Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-20T20:28:46.174267636Z Gathering data for ns/openshift-kube-scheduler...
2026-04-20T20:28:46.457934233Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-20T20:28:46.767324252Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-20T20:28:46.996440500Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-20T20:28:47.093384385Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-20T20:28:47.279761579Z Collecting performance related data for node ip-10-0-141-130.ec2.internal
2026-04-20T20:28:47.288318228Z Collecting performance related data for node ip-10-0-136-158.ec2.internal
2026-04-20T20:28:47.298388847Z Collecting performance related data for node ip-10-0-135-184.ec2.internal
2026-04-20T20:28:47.465662560Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-20T20:28:48.074890059Z Gathering data for ns/openshift-multus...
2026-04-20T20:28:49.459288208Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-20T20:28:49.898178852Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T20:28:50.957148013Z Gathering data for ns/openshift-host-network...
2026-04-20T20:28:51.242945553Z Gathering data for ns/openshift-network-diagnostics...
2026-04-20T20:28:51.768701362Z Gathering data for ns/openshift-network-node-identity...
2026-04-20T20:28:52.168003217Z Gathering data for ns/openshift-network-operator...
2026-04-20T20:28:52.585715495Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-20T20:28:52.861315408Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-20T20:28:53.311330345Z Gathering data for ns/openshift-apiserver-operator...
2026-04-20T20:28:53.641930624Z Gathering data for ns/openshift-apiserver...
2026-04-20T20:28:54.009374724Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-20T20:28:54.322346937Z Gathering data for ns/openshift-controller-manager...
2026-04-20T20:28:54.601607582Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-20T20:28:54.904903549Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T20:28:55.222635650Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-20T20:28:55.497130338Z Gathering data for ns/openshift-service-ca-operator...
2026-04-20T20:28:55.651340847Z Collecting kubelet logs for node ip-10-0-141-130.ec2.internal
2026-04-20T20:28:55.653221019Z Collecting kubelet logs for node ip-10-0-136-158.ec2.internal
2026-04-20T20:28:55.655101789Z Collecting kubelet logs for node ip-10-0-135-184.ec2.internal
2026-04-20T20:28:55.901239186Z Gathering data for ns/openshift-service-ca...
2026-04-20T20:28:56.034911612Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-20T20:28:56.039359606Z INFO: Node performance data collection complete.
2026-04-20T20:28:56.270376876Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-20T20:28:57.109035351Z Wrote inspect data to must-gather.
2026-04-20T20:28:57.109080218Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T20:28:57.109080218Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-20T20:28:57.244112321Z error: the server doesn't have a resource type "clusters"
2026-04-20T20:28:57.380206747Z Caches written to disk