2026-07-13T08:04:13.157687250Z [disk usage checker] Started
2026-07-13T08:04:13.161006933Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T08:04:13.353980230Z Gathering data for ns/openshift-cluster-version...
2026-07-13T08:04:13.480512569Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-13T08:04:13.558930197Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T08:04:13.638502182Z Gathering data for ns/default...
2026-07-13T08:04:13.916287714Z Gathering data for ns/openshift...
2026-07-13T08:04:14.316513307Z Gathering data for ns/kube-system...
2026-07-13T08:04:15.001474915Z Gathering data for ns/openshift-etcd...
2026-07-13T08:04:15.276567638Z Wrote inspect data to must-gather.
2026-07-13T08:04:15.276604278Z error: inspection completed with the errors occurred while gathering data:
2026-07-13T08:04:15.276604278Z     namespaces "assisted-installer" not found
2026-07-13T08:04:15.648449113Z Waiting on subprocesses to finish execution.
2026-07-13T08:04:15.649813663Z INFO: Gathering HAProxy config files
2026-07-13T08:04:15.656676805Z Executing Istio gather script
2026-07-13T08:04:15.660837803Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-13T08:04:15.667342477Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-13T08:04:15.672379840Z INFO: Collecting host service logs for crio
2026-07-13T08:04:15.672719889Z INFO: Collecting host service logs for kubelet
2026-07-13T08:04:15.673057706Z INFO: Collecting host service logs for rpm-ostreed
2026-07-13T08:04:15.673574453Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-13T08:04:15.673945425Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-13T08:04:15.674276033Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-13T08:04:15.674465885Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-13T08:04:15.674630064Z INFO: Collecting host service logs for NetworkManager
2026-07-13T08:04:15.674912564Z INFO: Collecting host service logs for openvswitch
2026-07-13T08:04:15.675221368Z INFO: Collecting host service logs for ovs-configuration
2026-07-13T08:04:15.675582584Z INFO: Collecting host service logs for ovsdb-server
2026-07-13T08:04:15.675875600Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-13T08:04:15.676252061Z INFO: Waiting for worker host service log collection to complete ...
2026-07-13T08:04:15.798079768Z INFO: Waiting for node performance related collection to complete ...
2026-07-13T08:04:16.135088689Z No resources found in openshift-etcd namespace.
2026-07-13T08:04:16.139907109Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-13T08:04:16.139944060Z INFO: on-disk MachineConfig config collection complete.
2026-07-13T08:04:16.391307522Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-13T08:04:16.504489088Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-13T08:04:16.507159649Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-4db4w
2026-07-13T08:04:16.507159649Z insights-runtime-extractor-crvfr
2026-07-13T08:04:16.507159649Z insights-runtime-extractor-pqrms
2026-07-13T08:04:16.507159649Z insights-runtime-extractor-rvhfc
2026-07-13T08:04:16.679169778Z INFO: "metallb-operator" not detected. Skipping.
2026-07-13T08:04:16.683090072Z No resources found
2026-07-13T08:04:16.874775976Z error: the server doesn't have a resource type "performanceprofile"
2026-07-13T08:04:16.932618922Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-13T08:04:16.972338447Z ERROR: No running kube-apiserver pods found
2026-07-13T08:04:16.996704702Z Wrote inspect data to must-gather.
2026-07-13T08:04:17.225307633Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-13T08:04:17.225307633Z See 'oc rsync -h' for help and examples
2026-07-13T08:04:17.330998229Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-13T08:04:17.364744585Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-13T08:04:17.460064775Z INFO: Worker host service log collection to complete.
2026-07-13T08:04:17.621248614Z Wrote inspect data to must-gather.
2026-07-13T08:04:17.671339267Z Wrote inspect data to must-gather.
2026-07-13T08:04:17.696244441Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-13T08:04:17.703532609Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-13T08:04:17.851634028Z Wrote inspect data to must-gather.
2026-07-13T08:04:17.906146230Z Gathering data for ns/openshift-monitoring...
2026-07-13T08:04:17.956240897Z No resources found
2026-07-13T08:04:17.979788342Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T08:04:18.089899967Z INFO: HAProxy config collection complete.
2026-07-13T08:04:18.122634872Z Wrote inspect data to must-gather.
2026-07-13T08:04:18.131410562Z INFO: 'previous-logs' folder not found on ip-10-0-135-11.ec2.internal, skipping...
2026-07-13T08:04:18.131630585Z Wrote inspect data to must-gather.
2026-07-13T08:04:18.171368601Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T08:04:18.260020843Z Wrote inspect data to must-gather.
2026-07-13T08:04:18.327106850Z error: the server doesn't have a resource type "machineconfigs"
2026-07-13T08:04:18.387324664Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-13T08:04:18.425787712Z INFO: 'previous-logs' folder not found on ip-10-0-135-15.ec2.internal, skipping...
2026-07-13T08:04:18.475367431Z INFO: Getting rules from prometheus-k8s-0
2026-07-13T08:04:18.514701646Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-13T08:04:18.562546177Z Wrote inspect data to must-gather.
2026-07-13T08:04:18.698945099Z INFO: 'previous-logs' folder not found on ip-10-0-143-113.ec2.internal, skipping...
2026-07-13T08:04:18.698976188Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-13T08:04:18.698983802Z INFO: Machine Config Daemon termination log collection complete.
2026-07-13T08:04:18.760520463Z Wrote inspect data to must-gather.
2026-07-13T08:04:18.824599525Z Wrote inspect data to must-gather.
2026-07-13T08:04:18.883913242Z INFO: Getting status/config from prometheus-k8s-0
2026-07-13T08:04:18.931965118Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-13T08:04:19.078548150Z Wrote inspect data to must-gather.
2026-07-13T08:04:19.128658858Z Wrote inspect data to must-gather.
2026-07-13T08:04:19.321541526Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-13T08:04:19.324799285Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-13T08:04:19.367288964Z Wrote inspect data to must-gather.
2026-07-13T08:04:19.384670683Z INFO: INTERCONNECT MODE
2026-07-13T08:04:19.384729936Z INFO: Gathering ovn-kubernetes DBs
2026-07-13T08:04:19.408074151Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-13T08:04:19.512450385Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:19.547408181Z INFO: Gathering OVN_Northbound from ovnkube-node-dsc8r...
2026-07-13T08:04:19.547617190Z INFO: Gathering OVN_Northbound from ovnkube-node-dwxl9...
2026-07-13T08:04:19.547870397Z INFO: Gathering OVN_Northbound from ovnkube-node-n9m6m...
2026-07-13T08:04:19.548185025Z INFO: Gathering OVN_Southbound from ovnkube-node-dsc8r...
2026-07-13T08:04:19.548496952Z INFO: Gathering OVN_Southbound from ovnkube-node-dwxl9...
2026-07-13T08:04:19.548779989Z INFO: Gathering OVN_Southbound from ovnkube-node-n9m6m...
2026-07-13T08:04:19.567718220Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-13T08:04:19.619211098Z Wrote inspect data to must-gather.
2026-07-13T08:04:19.887533437Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:19.894534737Z tar: Removing leading `/' from member names
2026-07-13T08:04:19.895230511Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-13T08:04:19.921002454Z tar: Removing leading `/' from member names
2026-07-13T08:04:19.925593715Z tar: Removing leading `/' from member names
2026-07-13T08:04:19.940461272Z tar: Removing leading `/' from member names
2026-07-13T08:04:19.968262006Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-13T08:04:20.003452381Z tar: Removing leading `/' from member names
2026-07-13T08:04:20.013297753Z tar: Removing leading `/' from member names
2026-07-13T08:04:20.014395953Z Wrote inspect data to must-gather.
2026-07-13T08:04:20.142934105Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:20.220767196Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T08:04:20.245985896Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-13T08:04:20.377742110Z Wrote inspect data to must-gather.
2026-07-13T08:04:20.391591419Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:20.396634109Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-13T08:04:20.452803467Z Gathering data for ns/openshift-network-console...
2026-07-13T08:04:20.494796125Z INFO: Waiting for network log collection to complete ...
2026-07-13T08:04:20.494840016Z INFO: Waiting for ovnk database copies to complete ...
2026-07-13T08:04:20.495256270Z INFO: Copying ovnk databases complete.
2026-07-13T08:04:20.498474247Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-13T08:04:20.502006443Z ovnk_database_store/
2026-07-13T08:04:20.502058709Z ovnk_database_store/ovnkube-node-dwxl9_nbdb
2026-07-13T08:04:20.512814986Z ovnk_database_store/ovnkube-node-dsc8r_nbdb
2026-07-13T08:04:20.520742056Z ovnk_database_store/ovnkube-node-dwxl9_sbdb
2026-07-13T08:04:20.539153382Z ovnk_database_store/ovnkube-node-n9m6m_nbdb
2026-07-13T08:04:20.547394849Z ovnk_database_store/ovnkube-node-dsc8r_sbdb
2026-07-13T08:04:20.564833868Z ovnk_database_store/ovnkube-node-n9m6m_sbdb
2026-07-13T08:04:20.587756207Z INFO: Network log collection complete.
2026-07-13T08:04:20.623776806Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:20.693831268Z INFO: Getting status from alertmanager-main-0
2026-07-13T08:04:20.721906673Z Wrote inspect data to must-gather.
2026-07-13T08:04:20.857583980Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:20.860886322Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-13T08:04:20.934091357Z Gathering data for ns/openshift-console-operator...
2026-07-13T08:04:21.039324016Z Wrote inspect data to must-gather.
2026-07-13T08:04:21.086715802Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:21.286795838Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-13T08:04:21.289557253Z INFO: Fallback to identify the container image from release info
2026-07-13T08:04:21.292229983Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:21.294993010Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-13T08:04:21.309115317Z Gathering data for ns/openshift-console...
2026-07-13T08:04:21.463567615Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:21.633101938Z Wrote inspect data to /must-gather/istio.
2026-07-13T08:04:21.780370467Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-13T08:04:21.902022717Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-13T08:04:22.025219817Z daemonset.apps/perf-node-gather-daemonset created
2026-07-13T08:04:22.167721883Z Waiting for performance profile collector pods to become ready: 1
2026-07-13T08:04:22.302546564Z Gathering data for ns/openshift-dns-operator...
2026-07-13T08:04:22.326603556Z Done executing Istio gather script
2026-07-13T08:04:22.575335281Z Gathering data for ns/openshift-dns...
2026-07-13T08:04:23.093130588Z Gathering data for ns/openshift-image-registry...
2026-07-13T08:04:23.176206139Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T08:04:23.283040255Z Waiting for performance profile collector pods to become ready: 2
2026-07-13T08:04:23.514928971Z Gathering data for ns/openshift-ingress-operator...
2026-07-13T08:04:23.837967262Z Gathering data for ns/openshift-ingress...
2026-07-13T08:04:24.142587502Z Gathering data for ns/openshift-ingress-canary...
2026-07-13T08:04:24.389571579Z Waiting for performance profile collector pods to become ready: 3
2026-07-13T08:04:24.506470379Z Gathering data for ns/openshift-insights...
2026-07-13T08:04:25.491299528Z Waiting for performance profile collector pods to become ready: 4
2026-07-13T08:04:26.246121050Z Gathering data for ns/openshift-config...
2026-07-13T08:04:26.545341394Z Gathering data for ns/openshift-config-managed...
2026-07-13T08:04:26.623722654Z Waiting for performance profile collector pods to become ready: 5
2026-07-13T08:04:26.928920708Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-13T08:04:27.159659416Z Gathering data for ns/openshift-kube-apiserver...
2026-07-13T08:04:27.381542869Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-13T08:04:27.654882582Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-13T08:04:27.748918835Z Waiting for performance profile collector pods to become ready: 6
2026-07-13T08:04:27.925968900Z Gathering data for ns/openshift-kube-scheduler...
2026-07-13T08:04:28.154089033Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-13T08:04:28.182332652Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T08:04:28.410609939Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-13T08:04:28.707840290Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-13T08:04:28.857962139Z Waiting for performance profile collector pods to become ready: 7
2026-07-13T08:04:28.999415719Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-13T08:04:29.480531666Z Gathering data for ns/openshift-multus...
2026-07-13T08:04:29.960141870Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-13T08:04:30.184312031Z Collecting performance related data for node ip-10-0-143-113.ec2.internal
2026-07-13T08:04:30.191810582Z Collecting performance related data for node ip-10-0-135-15.ec2.internal
2026-07-13T08:04:30.200327744Z Collecting performance related data for node ip-10-0-135-11.ec2.internal
2026-07-13T08:04:30.895397043Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-13T08:04:32.095780494Z Gathering data for ns/openshift-host-network...
2026-07-13T08:04:32.374493184Z Gathering data for ns/openshift-network-diagnostics...
2026-07-13T08:04:32.901766366Z Gathering data for ns/openshift-network-node-identity...
2026-07-13T08:04:33.190597599Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T08:04:33.316970560Z Gathering data for ns/openshift-network-operator...
2026-07-13T08:04:33.662250980Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-13T08:04:33.907940788Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-13T08:04:34.364599295Z Gathering data for ns/openshift-apiserver-operator...
2026-07-13T08:04:34.645053451Z Gathering data for ns/openshift-apiserver...
2026-07-13T08:04:35.024199137Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-13T08:04:35.308727764Z Gathering data for ns/openshift-controller-manager...
2026-07-13T08:04:35.570151560Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-13T08:04:36.202986847Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-13T08:04:36.506673249Z Gathering data for ns/openshift-service-ca-operator...
2026-07-13T08:04:36.789264616Z Gathering data for ns/openshift-service-ca...
2026-07-13T08:04:37.117464114Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-13T08:04:37.975560909Z Wrote inspect data to must-gather.
2026-07-13T08:04:37.975610032Z error: inspection completed with the errors occurred while gathering data:
2026-07-13T08:04:37.975610032Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-13T08:04:38.195369879Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T08:04:38.284669066Z Collecting kubelet logs for node ip-10-0-143-113.ec2.internal
2026-07-13T08:04:38.286303386Z Collecting kubelet logs for node ip-10-0-135-15.ec2.internal
2026-07-13T08:04:38.287997345Z Collecting kubelet logs for node ip-10-0-135-11.ec2.internal
2026-07-13T08:04:38.568744943Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-jg8w6 namespace
2026-07-13T08:04:38.573750494Z INFO: Node performance data collection complete.
2026-07-13T08:04:38.692181825Z error: the server doesn't have a resource type "clusters"
2026-07-13T08:04:38.844614047Z Caches written to disk