2026-07-16T18:01:17.852912988Z [disk usage checker] Started
2026-07-16T18:01:17.856110885Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-16T18:01:18.046425403Z Gathering data for ns/openshift-cluster-version...
2026-07-16T18:01:18.191751507Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-16T18:01:18.290626467Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-16T18:01:18.373866668Z Gathering data for ns/default...
2026-07-16T18:01:18.665577206Z Gathering data for ns/openshift...
2026-07-16T18:01:19.101968596Z Gathering data for ns/kube-system...
2026-07-16T18:01:19.860877668Z Gathering data for ns/openshift-etcd...
2026-07-16T18:01:20.185421280Z Wrote inspect data to must-gather.
2026-07-16T18:01:20.185529999Z error: inspection completed with the errors occurred while gathering data:
2026-07-16T18:01:20.185529999Z     namespaces "assisted-installer" not found
2026-07-16T18:01:20.414723277Z Waiting on subprocesses to finish execution.
2026-07-16T18:01:20.420241582Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-16T18:01:20.423014527Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-16T18:01:20.448045366Z INFO: Gathering HAProxy config files
2026-07-16T18:01:20.465712419Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-16T18:01:20.469090706Z Executing Istio gather script
2026-07-16T18:01:20.497953610Z INFO: Waiting for node performance related collection to complete ...
2026-07-16T18:01:20.503043113Z INFO: Collecting host service logs for crio
2026-07-16T18:01:20.503474986Z INFO: Collecting host service logs for kubelet
2026-07-16T18:01:20.503826147Z INFO: Collecting host service logs for rpm-ostreed
2026-07-16T18:01:20.504178165Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-16T18:01:20.504533901Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-16T18:01:20.504887310Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-16T18:01:20.505241175Z INFO: Collecting host service logs for NetworkManager
2026-07-16T18:01:20.505584190Z INFO: Collecting host service logs for openvswitch
2026-07-16T18:01:20.505919368Z INFO: Collecting host service logs for ovs-configuration
2026-07-16T18:01:20.506285670Z INFO: Collecting host service logs for ovsdb-server
2026-07-16T18:01:20.506618739Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-16T18:01:20.507032783Z INFO: Waiting for worker host service log collection to complete ...
2026-07-16T18:01:20.867197835Z INFO: "metallb-operator" not detected. Skipping.
2026-07-16T18:01:21.226664389Z No resources found in openshift-etcd namespace.
2026-07-16T18:01:21.228051731Z No resources found
2026-07-16T18:01:21.235101022Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-mbmr9
2026-07-16T18:01:21.235101022Z insights-runtime-extractor-9dg4g
2026-07-16T18:01:21.235101022Z insights-runtime-extractor-hmprf
2026-07-16T18:01:21.235101022Z insights-runtime-extractor-m7pfv
2026-07-16T18:01:21.235879570Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-16T18:01:21.453223191Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-16T18:01:21.499562534Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-16T18:01:21.499562534Z See 'oc rsync -h' for help and examples
2026-07-16T18:01:21.522464081Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-16T18:01:21.530140883Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-16T18:01:21.612688386Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-16T18:01:21.612688386Z INFO: on-disk MachineConfig config collection complete.
2026-07-16T18:01:21.744802681Z Wrote inspect data to must-gather.
2026-07-16T18:01:21.862611985Z INFO: Worker host service log collection to complete.
2026-07-16T18:01:21.926161636Z error: the server doesn't have a resource type "performanceprofile"
2026-07-16T18:01:21.971733595Z ERROR: No running kube-apiserver pods found
2026-07-16T18:01:22.024131928Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-16T18:01:22.050444269Z Wrote inspect data to must-gather.
2026-07-16T18:01:22.221368743Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-16T18:01:22.406174833Z INFO: 'previous-logs' folder not found on ip-10-0-131-83.ec2.internal, skipping...
2026-07-16T18:01:22.502629570Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-16T18:01:22.552471397Z No resources found
2026-07-16T18:01:22.593411877Z Gathering data for ns/openshift-monitoring...
2026-07-16T18:01:22.613539624Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-16T18:01:22.862209480Z tar: Removing leading `/' from member names
2026-07-16T18:01:22.869523206Z INFO: HAProxy config collection complete.
2026-07-16T18:01:22.879971775Z Wrote inspect data to must-gather.
2026-07-16T18:01:22.921623781Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-16T18:01:22.948759756Z Wrote inspect data to must-gather.
2026-07-16T18:01:22.957499029Z Wrote inspect data to must-gather.
2026-07-16T18:01:22.974488047Z Wrote inspect data to must-gather.
2026-07-16T18:01:23.091230666Z INFO: 'previous-logs' folder not found on ip-10-0-137-190.ec2.internal, skipping...
2026-07-16T18:01:23.097669791Z Wrote inspect data to must-gather.
2026-07-16T18:01:23.105758488Z INFO: Getting rules from prometheus-k8s-0
2026-07-16T18:01:23.265388138Z Wrote inspect data to must-gather.
2026-07-16T18:01:23.280247931Z error: the server doesn't have a resource type "machineconfigs"
2026-07-16T18:01:23.308906006Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-16T18:01:23.361698970Z INFO: 'previous-logs' folder not found on ip-10-0-138-109.ec2.internal, skipping...
2026-07-16T18:01:23.361745624Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-16T18:01:23.361758665Z INFO: Machine Config Daemon termination log collection complete.
2026-07-16T18:01:23.431021088Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-16T18:01:23.504673477Z Wrote inspect data to must-gather.
2026-07-16T18:01:23.580118295Z INFO: Getting status/config from prometheus-k8s-0
2026-07-16T18:01:23.628297401Z Wrote inspect data to must-gather.
2026-07-16T18:01:23.759792957Z Wrote inspect data to must-gather.
2026-07-16T18:01:23.788189428Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-16T18:01:23.991112299Z Wrote inspect data to must-gather.
2026-07-16T18:01:24.006502385Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-16T18:01:24.045958565Z INFO: INTERCONNECT MODE
2026-07-16T18:01:24.046022122Z INFO: Gathering ovn-kubernetes DBs
2026-07-16T18:01:24.093880306Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-16T18:01:24.198941520Z Wrote inspect data to must-gather.
2026-07-16T18:01:24.204823963Z INFO: Gathering OVN_Northbound from ovnkube-node-k954t...
2026-07-16T18:01:24.205043060Z INFO: Gathering OVN_Northbound from ovnkube-node-kk57w...
2026-07-16T18:01:24.205233834Z INFO: Gathering OVN_Northbound from ovnkube-node-wzkkw...
2026-07-16T18:01:24.205480268Z INFO: Gathering OVN_Southbound from ovnkube-node-k954t...
2026-07-16T18:01:24.205732569Z INFO: Gathering OVN_Southbound from ovnkube-node-kk57w...
2026-07-16T18:01:24.206056999Z INFO: Gathering OVN_Southbound from ovnkube-node-wzkkw...
2026-07-16T18:01:24.281097981Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-16T18:01:24.314851823Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:24.520153692Z tar: Removing leading `/' from member names
2026-07-16T18:01:24.526028771Z tar: Removing leading `/' from member names
2026-07-16T18:01:24.541229819Z Wrote inspect data to must-gather.
2026-07-16T18:01:24.550843099Z tar: Removing leading `/' from member names
2026-07-16T18:01:24.560779742Z tar: Removing leading `/' from member names
2026-07-16T18:01:24.582638952Z tar: Removing leading `/' from member names
2026-07-16T18:01:24.590953189Z tar: Removing leading `/' from member names
2026-07-16T18:01:24.630424054Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:24.634297347Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-16T18:01:24.676961507Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-16T18:01:24.787038257Z Wrote inspect data to must-gather.
2026-07-16T18:01:24.879411440Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:24.917490680Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-16T18:01:25.019333404Z Wrote inspect data to must-gather.
2026-07-16T18:01:25.042779256Z INFO: Waiting for network log collection to complete ...
2026-07-16T18:01:25.042911738Z INFO: Waiting for ovnk database copies to complete ...
2026-07-16T18:01:25.043534477Z INFO: Copying ovnk databases complete.
2026-07-16T18:01:25.045695940Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-07-16T18:01:25.051859811Z ovnk_database_store/
2026-07-16T18:01:25.051859811Z ovnk_database_store/ovnkube-node-wzkkw_nbdb
2026-07-16T18:01:25.052206745Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-16T18:01:25.064164150Z ovnk_database_store/ovnkube-node-k954t_nbdb
2026-07-16T18:01:25.075653580Z ovnk_database_store/ovnkube-node-wzkkw_sbdb
2026-07-16T18:01:25.101077564Z ovnk_database_store/ovnkube-node-kk57w_sbdb
2026-07-16T18:01:25.112846588Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:25.115181384Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-16T18:01:25.130251129Z ovnk_database_store/ovnkube-node-kk57w_nbdb
2026-07-16T18:01:25.140276905Z ovnk_database_store/ovnkube-node-k954t_sbdb
2026-07-16T18:01:25.164982689Z INFO: Network log collection complete.
2026-07-16T18:01:25.235582635Z Wrote inspect data to must-gather.
2026-07-16T18:01:25.318834881Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:25.458644858Z Wrote inspect data to must-gather.
2026-07-16T18:01:25.473867229Z INFO: Getting status from alertmanager-main-0
2026-07-16T18:01:25.517296834Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:25.520325963Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-16T18:01:25.615249368Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-16T18:01:25.751292557Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-16T18:01:25.754778718Z INFO: Fallback to identify the container image from release info
2026-07-16T18:01:25.772101906Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:25.809090057Z Gathering data for ns/openshift-network-console...
2026-07-16T18:01:25.965821392Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:25.968044418Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-16T18:01:26.115388271Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:26.307729954Z Wrote inspect data to /must-gather/istio.
2026-07-16T18:01:26.309890614Z Gathering data for ns/openshift-console-operator...
2026-07-16T18:01:26.553126504Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-16T18:01:26.681811926Z daemonset.apps/perf-node-gather-daemonset created
2026-07-16T18:01:26.783197900Z Gathering data for ns/openshift-console...
2026-07-16T18:01:26.806833020Z Waiting for performance profile collector pods to become ready: 1
2026-07-16T18:01:26.981819483Z Done executing Istio gather script
2026-07-16T18:01:27.361002698Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-16T18:01:27.920934437Z Gathering data for ns/openshift-dns-operator...
2026-07-16T18:01:27.925797125Z Waiting for performance profile collector pods to become ready: 2
2026-07-16T18:01:27.930162382Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-16T18:01:28.222178120Z Gathering data for ns/openshift-dns...
2026-07-16T18:01:28.853249725Z Gathering data for ns/openshift-image-registry...
2026-07-16T18:01:29.040184790Z Waiting for performance profile collector pods to become ready: 3
2026-07-16T18:01:29.378046325Z Gathering data for ns/openshift-ingress-operator...
2026-07-16T18:01:29.709697593Z Gathering data for ns/openshift-ingress...
2026-07-16T18:01:30.093542926Z Gathering data for ns/openshift-ingress-canary...
2026-07-16T18:01:30.149730312Z Waiting for performance profile collector pods to become ready: 4
2026-07-16T18:01:30.494252815Z Gathering data for ns/openshift-insights...
2026-07-16T18:01:31.261794958Z Waiting for performance profile collector pods to become ready: 5
2026-07-16T18:01:32.368862916Z Waiting for performance profile collector pods to become ready: 6
2026-07-16T18:01:32.581241842Z Gathering data for ns/openshift-config...
2026-07-16T18:01:32.934485930Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-16T18:01:32.956699602Z Gathering data for ns/openshift-config-managed...
2026-07-16T18:01:33.395350275Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-16T18:01:33.495413926Z Waiting for performance profile collector pods to become ready: 7
2026-07-16T18:01:33.679669574Z Gathering data for ns/openshift-kube-apiserver...
2026-07-16T18:01:33.991227951Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-16T18:01:34.302823535Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-16T18:01:34.599260202Z Gathering data for ns/openshift-kube-scheduler...
2026-07-16T18:01:34.616532269Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-16T18:01:34.874440912Z Collecting performance related data for node ip-10-0-137-190.ec2.internal
2026-07-16T18:01:34.880425181Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-16T18:01:34.881881001Z Collecting performance related data for node ip-10-0-138-109.ec2.internal
2026-07-16T18:01:34.889743142Z Collecting performance related data for node ip-10-0-131-83.ec2.internal
2026-07-16T18:01:35.250779886Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-16T18:01:35.639813599Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-16T18:01:36.066890556Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-16T18:01:36.669545640Z Gathering data for ns/openshift-multus...
2026-07-16T18:01:37.941196360Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-16T18:01:38.407131665Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-16T18:01:39.720468966Z Gathering data for ns/openshift-host-network...
2026-07-16T18:01:40.108494682Z Gathering data for ns/openshift-network-diagnostics...
2026-07-16T18:01:40.673921498Z Gathering data for ns/openshift-network-node-identity...
2026-07-16T18:01:41.082212972Z Gathering data for ns/openshift-network-operator...
2026-07-16T18:01:41.462928075Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-16T18:01:41.722807522Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-16T18:01:42.333228950Z Gathering data for ns/openshift-apiserver-operator...
2026-07-16T18:01:42.671673752Z Gathering data for ns/openshift-apiserver...
2026-07-16T18:01:42.937931855Z Collecting kubelet logs for node ip-10-0-137-190.ec2.internal
2026-07-16T18:01:42.940846105Z Collecting kubelet logs for node ip-10-0-138-109.ec2.internal
2026-07-16T18:01:42.943115235Z Collecting kubelet logs for node ip-10-0-131-83.ec2.internal
2026-07-16T18:01:42.946694006Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-16T18:01:43.095423298Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-16T18:01:43.294418479Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-bn77j namespace
2026-07-16T18:01:43.300563048Z INFO: Node performance data collection complete.
2026-07-16T18:01:43.384911315Z Gathering data for ns/openshift-controller-manager...
2026-07-16T18:01:43.676366615Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-16T18:01:44.317545924Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-16T18:01:44.587392273Z Gathering data for ns/openshift-service-ca-operator...
2026-07-16T18:01:44.899663209Z Gathering data for ns/openshift-service-ca...
2026-07-16T18:01:45.263377582Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-16T18:01:46.163948406Z Wrote inspect data to must-gather.
2026-07-16T18:01:46.164013394Z error: inspection completed with the errors occurred while gathering data:
2026-07-16T18:01:46.164013394Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-16T18:01:46.286601033Z error: the server doesn't have a resource type "clusters"
2026-07-16T18:01:46.406630727Z Caches written to disk