2026-07-30T13:52:11.792227578Z [disk usage checker] Started
2026-07-30T13:52:11.795885381Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-30T13:52:11.993931837Z Gathering data for ns/openshift-cluster-version...
2026-07-30T13:52:12.126280031Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T13:52:12.218719257Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T13:52:12.307815618Z Gathering data for ns/default...
2026-07-30T13:52:12.618257715Z Gathering data for ns/openshift...
2026-07-30T13:52:13.040703975Z Gathering data for ns/kube-system...
2026-07-30T13:52:13.748696279Z Gathering data for ns/openshift-etcd...
2026-07-30T13:52:14.073351916Z Wrote inspect data to must-gather.
2026-07-30T13:52:14.073384599Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T13:52:14.073384599Z     namespaces "assisted-installer" not found
2026-07-30T13:52:14.412809301Z Waiting on subprocesses to finish execution.
2026-07-30T13:52:14.421763615Z Executing Istio gather script
2026-07-30T13:52:14.430850703Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-30T13:52:14.436941968Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-30T13:52:14.442799178Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-30T13:52:14.456267227Z INFO: Collecting host service logs for crio
2026-07-30T13:52:14.456579286Z INFO: Collecting host service logs for kubelet
2026-07-30T13:52:14.456914877Z INFO: Collecting host service logs for rpm-ostreed
2026-07-30T13:52:14.457205836Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-30T13:52:14.457495957Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-30T13:52:14.457813215Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-30T13:52:14.458121243Z INFO: Collecting host service logs for NetworkManager
2026-07-30T13:52:14.458437286Z INFO: Collecting host service logs for openvswitch
2026-07-30T13:52:14.458761617Z INFO: Collecting host service logs for ovs-configuration
2026-07-30T13:52:14.459266159Z INFO: Collecting host service logs for ovsdb-server
2026-07-30T13:52:14.459590031Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-30T13:52:14.459985695Z INFO: Waiting for worker host service log collection to complete ...
2026-07-30T13:52:14.478599492Z INFO: Gathering HAProxy config files
2026-07-30T13:52:14.615496742Z INFO: Waiting for node performance related collection to complete ...
2026-07-30T13:52:14.836094238Z INFO: "metallb-operator" not detected. Skipping.
2026-07-30T13:52:15.029767046Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-30T13:52:15.052814536Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-30T13:52:15.158403405Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-30T13:52:15.702304665Z No resources found in openshift-etcd namespace.
2026-07-30T13:52:15.829102461Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T13:52:15.835242787Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-30T13:52:15.835242787Z INFO: on-disk MachineConfig config collection complete.
2026-07-30T13:52:15.850864025Z error: the server doesn't have a resource type "performanceprofile"
2026-07-30T13:52:15.995424730Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-xhcv5
2026-07-30T13:52:15.995489876Z insights-runtime-extractor-9jlvw
2026-07-30T13:52:15.995509329Z insights-runtime-extractor-dk7qz
2026-07-30T13:52:15.995528137Z insights-runtime-extractor-rkd7d
2026-07-30T13:52:16.092945210Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-30T13:52:16.162345601Z Wrote inspect data to must-gather.
2026-07-30T13:52:16.210842617Z ERROR: No running kube-apiserver pods found
2026-07-30T13:52:16.235610764Z No resources found
2026-07-30T13:52:16.323134624Z INFO: Worker host service log collection to complete.
2026-07-30T13:52:16.466367268Z No resources found
2026-07-30T13:52:16.553922886Z Wrote inspect data to must-gather.
2026-07-30T13:52:16.589977185Z Wrote inspect data to must-gather.
2026-07-30T13:52:16.591890515Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-30T13:52:16.646274469Z Wrote inspect data to must-gather.
2026-07-30T13:52:16.704962717Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-30T13:52:16.721556189Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-30T13:52:16.721556189Z See 'oc rsync -h' for help and examples
2026-07-30T13:52:16.772036698Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-30T13:52:16.804542359Z Wrote inspect data to must-gather.
2026-07-30T13:52:16.810188030Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-30T13:52:16.828490683Z Wrote inspect data to must-gather.
2026-07-30T13:52:16.982058764Z Wrote inspect data to must-gather.
2026-07-30T13:52:16.986410042Z INFO: 'previous-logs' folder not found on ip-10-0-129-21.ec2.internal, skipping...
2026-07-30T13:52:17.138755960Z tar: Removing leading `/' from member names
2026-07-30T13:52:17.140393017Z Wrote inspect data to must-gather.
2026-07-30T13:52:17.148270299Z INFO: HAProxy config collection complete.
2026-07-30T13:52:17.151939112Z error: the server doesn't have a resource type "machineconfigs"
2026-07-30T13:52:17.226510015Z Gathering data for ns/openshift-monitoring...
2026-07-30T13:52:17.418353410Z INFO: Getting rules from prometheus-k8s-0
2026-07-30T13:52:17.421100012Z INFO: 'previous-logs' folder not found on ip-10-0-132-205.ec2.internal, skipping...
2026-07-30T13:52:17.423608375Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-30T13:52:17.585852272Z Wrote inspect data to must-gather.
2026-07-30T13:52:17.781049463Z Wrote inspect data to must-gather.
2026-07-30T13:52:17.814553197Z INFO: 'previous-logs' folder not found on ip-10-0-140-162.ec2.internal, skipping...
2026-07-30T13:52:17.814583602Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-30T13:52:17.814616048Z INFO: Machine Config Daemon termination log collection complete.
2026-07-30T13:52:17.855792628Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-30T13:52:17.900687399Z Wrote inspect data to must-gather.
2026-07-30T13:52:17.974220865Z INFO: Getting status/config from prometheus-k8s-0
2026-07-30T13:52:17.982081640Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-30T13:52:18.217978972Z INFO: INTERCONNECT MODE
2026-07-30T13:52:18.218008589Z INFO: Gathering ovn-kubernetes DBs
2026-07-30T13:52:18.222401930Z Wrote inspect data to must-gather.
2026-07-30T13:52:18.369305855Z INFO: Gathering OVN_Northbound from ovnkube-node-9dsmv...
2026-07-30T13:52:18.369562311Z INFO: Gathering OVN_Northbound from ovnkube-node-hdq67...
2026-07-30T13:52:18.369830850Z INFO: Gathering OVN_Northbound from ovnkube-node-t2x4t...
2026-07-30T13:52:18.370136849Z INFO: Gathering OVN_Southbound from ovnkube-node-9dsmv...
2026-07-30T13:52:18.370381584Z INFO: Gathering OVN_Southbound from ovnkube-node-hdq67...
2026-07-30T13:52:18.370690795Z INFO: Gathering OVN_Southbound from ovnkube-node-t2x4t...
2026-07-30T13:52:18.402996081Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-30T13:52:18.469005122Z Wrote inspect data to must-gather.
2026-07-30T13:52:18.586607746Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-30T13:52:18.684352035Z tar: Removing leading `/' from member names
2026-07-30T13:52:18.707299414Z tar: Removing leading `/' from member names
2026-07-30T13:52:18.722821436Z tar: Removing leading `/' from member names
2026-07-30T13:52:18.729750372Z tar: Removing leading `/' from member names
2026-07-30T13:52:18.732508909Z tar: Removing leading `/' from member names
2026-07-30T13:52:18.735807578Z tar: Removing leading `/' from member names
2026-07-30T13:52:18.804733828Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-30T13:52:18.830360344Z Wrote inspect data to must-gather.
2026-07-30T13:52:18.833458830Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:18.988359733Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T13:52:19.088031533Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-30T13:52:19.093196107Z Wrote inspect data to must-gather.
2026-07-30T13:52:19.095505285Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:19.097786915Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-30T13:52:19.237820098Z INFO: Waiting for network log collection to complete ...
2026-07-30T13:52:19.237944017Z INFO: Waiting for ovnk database copies to complete ...
2026-07-30T13:52:19.238471063Z INFO: Copying ovnk databases complete.
2026-07-30T13:52:19.241756782Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-30T13:52:19.245159229Z ovnk_database_store/
2026-07-30T13:52:19.245211882Z ovnk_database_store/ovnkube-node-t2x4t_sbdb
2026-07-30T13:52:19.265141424Z ovnk_database_store/ovnkube-node-9dsmv_sbdb
2026-07-30T13:52:19.282813646Z ovnk_database_store/ovnkube-node-hdq67_sbdb
2026-07-30T13:52:19.297860147Z ovnk_database_store/ovnkube-node-9dsmv_nbdb
2026-07-30T13:52:19.304753834Z ovnk_database_store/ovnkube-node-hdq67_nbdb
2026-07-30T13:52:19.312245117Z ovnk_database_store/ovnkube-node-t2x4t_nbdb
2026-07-30T13:52:19.322199285Z INFO: Network log collection complete.
2026-07-30T13:52:19.343067743Z Wrote inspect data to must-gather.
2026-07-30T13:52:19.344553302Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:19.370411611Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-30T13:52:19.547562462Z W0730 13:52:19.547511    1505 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-30T13:52:19.553872309Z Wrote inspect data to must-gather.
2026-07-30T13:52:19.574599528Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:19.578722384Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-30T13:52:19.661296991Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T13:52:19.758162970Z INFO: Getting status from alertmanager-main-0
2026-07-30T13:52:19.803408102Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:19.844565203Z Wrote inspect data to must-gather.
2026-07-30T13:52:19.860817570Z Gathering data for ns/openshift-network-console...
2026-07-30T13:52:20.035033402Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:20.038481033Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-30T13:52:20.113568805Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-30T13:52:20.116093463Z INFO: Fallback to identify the container image from release info
2026-07-30T13:52:20.266697433Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:20.314881039Z Gathering data for ns/openshift-console-operator...
2026-07-30T13:52:20.454788877Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:20.457379807Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-30T13:52:20.607820477Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:20.730932325Z Gathering data for ns/openshift-console...
2026-07-30T13:52:20.804614081Z Wrote inspect data to /must-gather/istio.
2026-07-30T13:52:20.806066614Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-30T13:52:20.948783177Z daemonset.apps/perf-node-gather-daemonset created
2026-07-30T13:52:21.080780128Z Waiting for performance profile collector pods to become ready: 1
2026-07-30T13:52:21.406171161Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-30T13:52:21.520332295Z Done executing Istio gather script
2026-07-30T13:52:21.816337146Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-30T13:52:21.922119064Z Gathering data for ns/openshift-dns-operator...
2026-07-30T13:52:22.210811050Z Waiting for performance profile collector pods to become ready: 2
2026-07-30T13:52:22.249501049Z Gathering data for ns/openshift-dns...
2026-07-30T13:52:22.829964146Z Gathering data for ns/openshift-image-registry...
2026-07-30T13:52:23.307454239Z Gathering data for ns/openshift-ingress-operator...
2026-07-30T13:52:23.327499093Z Waiting for performance profile collector pods to become ready: 3
2026-07-30T13:52:23.632996454Z Gathering data for ns/openshift-ingress...
2026-07-30T13:52:23.953396075Z Gathering data for ns/openshift-ingress-canary...
2026-07-30T13:52:24.328034383Z Gathering data for ns/openshift-insights...
2026-07-30T13:52:24.433066723Z Waiting for performance profile collector pods to become ready: 4
2026-07-30T13:52:25.540074373Z Waiting for performance profile collector pods to become ready: 5
2026-07-30T13:52:26.243533061Z Gathering data for ns/openshift-jobset-operator...
2026-07-30T13:52:26.668658526Z Waiting for performance profile collector pods to become ready: 6
2026-07-30T13:52:26.821344352Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-30T13:52:27.038327863Z Gathering data for ns/openshift-config...
2026-07-30T13:52:27.298009828Z Gathering data for ns/openshift-config-managed...
2026-07-30T13:52:27.732157133Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-30T13:52:27.798721776Z Waiting for performance profile collector pods to become ready: 7
2026-07-30T13:52:28.001801661Z Gathering data for ns/openshift-kube-apiserver...
2026-07-30T13:52:28.262653166Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-30T13:52:28.555641058Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-30T13:52:28.913361333Z Gathering data for ns/openshift-kube-scheduler...
2026-07-30T13:52:28.916102107Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-30T13:52:29.169161626Z Collecting performance related data for node ip-10-0-140-162.ec2.internal
2026-07-30T13:52:29.179805077Z Collecting performance related data for node ip-10-0-132-205.ec2.internal
2026-07-30T13:52:29.189232246Z Collecting performance related data for node ip-10-0-129-21.ec2.internal
2026-07-30T13:52:29.189775289Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-30T13:52:29.540834564Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-30T13:52:29.918777486Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-30T13:52:30.304192061Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-30T13:52:30.901309321Z Gathering data for ns/openshift-multus...
2026-07-30T13:52:31.826212277Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-30T13:52:32.352391090Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-30T13:52:33.555862077Z Gathering data for ns/openshift-host-network...
2026-07-30T13:52:33.913026648Z Gathering data for ns/openshift-network-diagnostics...
2026-07-30T13:52:34.450884589Z Gathering data for ns/openshift-network-node-identity...
2026-07-30T13:52:34.869870930Z Gathering data for ns/openshift-network-operator...
2026-07-30T13:52:35.243312681Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-30T13:52:35.503955111Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-30T13:52:36.008305512Z Gathering data for ns/openshift-apiserver-operator...
2026-07-30T13:52:36.318762383Z Gathering data for ns/openshift-apiserver...
2026-07-30T13:52:36.738066174Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-30T13:52:36.830884863Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-30T13:52:37.040524308Z Gathering data for ns/openshift-controller-manager...
2026-07-30T13:52:37.354941753Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-30T13:52:37.699581624Z Collecting kubelet logs for node ip-10-0-140-162.ec2.internal
2026-07-30T13:52:37.702173283Z Collecting kubelet logs for node ip-10-0-132-205.ec2.internal
2026-07-30T13:52:37.705110922Z Collecting kubelet logs for node ip-10-0-129-21.ec2.internal
2026-07-30T13:52:38.064613632Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-699cb namespace
2026-07-30T13:52:38.070989718Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-30T13:52:38.071651101Z INFO: Node performance data collection complete.
2026-07-30T13:52:38.417224252Z Gathering data for ns/openshift-service-ca-operator...
2026-07-30T13:52:38.732457997Z Gathering data for ns/openshift-service-ca...
2026-07-30T13:52:39.095645651Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-30T13:52:39.957814426Z Wrote inspect data to must-gather.
2026-07-30T13:52:39.957854898Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T13:52:39.957854898Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-30T13:52:40.089112360Z error: the server doesn't have a resource type "clusters"
2026-07-30T13:52:40.230998964Z Caches written to disk