2026-08-05T15:51:07.785376407Z [disk usage checker] Started
2026-08-05T15:51:07.788667203Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-05T15:51:07.976252034Z Gathering data for ns/openshift-cluster-version...
2026-08-05T15:51:08.117649992Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-05T15:51:08.197951944Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-05T15:51:08.275592434Z Gathering data for ns/default...
2026-08-05T15:51:08.535957908Z Gathering data for ns/openshift...
2026-08-05T15:51:08.920023898Z Gathering data for ns/kube-system...
2026-08-05T15:51:09.581974106Z Gathering data for ns/openshift-etcd...
2026-08-05T15:51:09.881725115Z Wrote inspect data to must-gather.
2026-08-05T15:51:09.881756969Z error: inspection completed with the errors occurred while gathering data:
2026-08-05T15:51:09.881756969Z     namespaces "assisted-installer" not found
2026-08-05T15:51:10.231270452Z Waiting on subprocesses to finish execution.
2026-08-05T15:51:10.239475624Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-08-05T15:51:10.242714427Z Executing Istio gather script
2026-08-05T15:51:10.248392807Z INFO: Gathering HAProxy config files
2026-08-05T15:51:10.249780556Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-08-05T15:51:10.250019769Z INFO: Gathering machine config daemon's old logs from all nodes
2026-08-05T15:51:10.279863446Z INFO: Waiting for node performance related collection to complete ...
2026-08-05T15:51:10.289013088Z INFO: Collecting host service logs for crio
2026-08-05T15:51:10.289387600Z INFO: Collecting host service logs for kubelet
2026-08-05T15:51:10.289762882Z INFO: Collecting host service logs for rpm-ostreed
2026-08-05T15:51:10.290125291Z INFO: Collecting host service logs for ostree-finalize-staged
2026-08-05T15:51:10.290483034Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-08-05T15:51:10.290837274Z INFO: Collecting host service logs for machine-config-daemon-host
2026-08-05T15:51:10.291188455Z INFO: Collecting host service logs for NetworkManager
2026-08-05T15:51:10.291546353Z INFO: Collecting host service logs for openvswitch
2026-08-05T15:51:10.291963904Z INFO: Collecting host service logs for ovs-configuration
2026-08-05T15:51:10.292341608Z INFO: Collecting host service logs for ovsdb-server
2026-08-05T15:51:10.292704043Z INFO: Collecting host service logs for ovs-vswitchd
2026-08-05T15:51:10.293118806Z INFO: Waiting for worker host service log collection to complete ...
2026-08-05T15:51:10.644555838Z INFO: "metallb-operator" not detected. Skipping.
2026-08-05T15:51:11.025611390Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-08-05T15:51:11.054889377Z INFO: Found 1 replicas - prometheus-k8s-0
2026-08-05T15:51:11.177261549Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-08-05T15:51:11.177347007Z INFO: on-disk MachineConfig config collection complete.
2026-08-05T15:51:11.362783766Z Wrote inspect data to must-gather.
2026-08-05T15:51:11.387350818Z No resources found
2026-08-05T15:51:11.409451570Z INFO: "sriov-network-operator" not detected. Skipping.
2026-08-05T15:51:11.416095630Z INFO: Collecting Insights Archives from insights-operator-77696685b-phz7t
2026-08-05T15:51:11.416095630Z insights-runtime-extractor-4mz4r
2026-08-05T15:51:11.416095630Z insights-runtime-extractor-fvt5r
2026-08-05T15:51:11.416095630Z insights-runtime-extractor-wfg2d
2026-08-05T15:51:11.497353608Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-08-05T15:51:11.641781549Z error: the server doesn't have a resource type "performanceprofile"
2026-08-05T15:51:11.660467207Z No resources found in openshift-etcd namespace.
2026-08-05T15:51:11.709952009Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-08-05T15:51:11.709952009Z See 'oc rsync -h' for help and examples
2026-08-05T15:51:11.766926681Z ERROR: No running kube-apiserver pods found
2026-08-05T15:51:11.847749020Z Wrote inspect data to must-gather.
2026-08-05T15:51:11.850889862Z INFO: Waiting for HAProxy config collection to complete ...
2026-08-05T15:51:11.871708772Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-08-05T15:51:11.929637451Z INFO: Worker host service log collection to complete.
2026-08-05T15:51:12.108026537Z INFO: 'previous-logs' folder not found on ip-10-0-135-205.ec2.internal, skipping...
2026-08-05T15:51:12.418691474Z Gathering data for ns/openshift-monitoring...
2026-08-05T15:51:12.451094070Z Wrote inspect data to must-gather.
2026-08-05T15:51:12.454899881Z Wrote inspect data to must-gather.
2026-08-05T15:51:12.491191408Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-08-05T15:51:12.576054653Z tar: Removing leading `/' from member names
2026-08-05T15:51:12.580696375Z INFO: HAProxy config collection complete.
2026-08-05T15:51:12.591541448Z Wrote inspect data to must-gather.
2026-08-05T15:51:12.608919942Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-05T15:51:12.647005800Z No resources found
2026-08-05T15:51:12.842651516Z INFO: Getting rules from prometheus-k8s-0
2026-08-05T15:51:12.847875010Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-05T15:51:12.854745348Z Wrote inspect data to must-gather.
2026-08-05T15:51:12.910443611Z error: the server doesn't have a resource type "machineconfigs"
2026-08-05T15:51:12.941213387Z Wrote inspect data to must-gather.
2026-08-05T15:51:12.962141985Z INFO: 'previous-logs' folder not found on ip-10-0-136-120.ec2.internal, skipping...
2026-08-05T15:51:13.056891418Z Wrote inspect data to must-gather.
2026-08-05T15:51:13.109749264Z error: the server doesn't have a resource type "machineconfigpools"
2026-08-05T15:51:13.200459494Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-08-05T15:51:13.233652846Z INFO: 'previous-logs' folder not found on ip-10-0-142-177.ec2.internal, skipping...
2026-08-05T15:51:13.233687651Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-08-05T15:51:13.233727441Z INFO: Machine Config Daemon termination log collection complete.
2026-08-05T15:51:13.303768856Z INFO: Getting status/config from prometheus-k8s-0
2026-08-05T15:51:13.326688257Z Wrote inspect data to must-gather.
2026-08-05T15:51:13.352381722Z Wrote inspect data to must-gather.
2026-08-05T15:51:13.506063777Z error: the server doesn't have a resource type "kubeletconfigs"
2026-08-05T15:51:13.542737911Z Wrote inspect data to must-gather.
2026-08-05T15:51:13.675272202Z INFO: Getting status/flags from prometheus-k8s-0
2026-08-05T15:51:13.733183958Z Wrote inspect data to must-gather.
2026-08-05T15:51:13.824143574Z INFO: INTERCONNECT MODE
2026-08-05T15:51:13.824184892Z INFO: Gathering ovn-kubernetes DBs
2026-08-05T15:51:13.908863972Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-08-05T15:51:13.951544643Z Wrote inspect data to must-gather.
2026-08-05T15:51:13.955248636Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-08-05T15:51:13.994076897Z INFO: Gathering OVN_Northbound from ovnkube-node-4pzgd...
2026-08-05T15:51:13.994400842Z INFO: Gathering OVN_Northbound from ovnkube-node-prz9p...
2026-08-05T15:51:13.994650590Z INFO: Gathering OVN_Northbound from ovnkube-node-s9x49...
2026-08-05T15:51:13.994925478Z INFO: Gathering OVN_Southbound from ovnkube-node-4pzgd...
2026-08-05T15:51:13.995241332Z INFO: Gathering OVN_Southbound from ovnkube-node-prz9p...
2026-08-05T15:51:13.995560691Z INFO: Gathering OVN_Southbound from ovnkube-node-s9x49...
2026-08-05T15:51:14.146001591Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-08-05T15:51:14.229985531Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-08-05T15:51:14.308259442Z tar: Removing leading `/' from member names
2026-08-05T15:51:14.327255904Z tar: Removing leading `/' from member names
2026-08-05T15:51:14.336364253Z Wrote inspect data to must-gather.
2026-08-05T15:51:14.346802207Z tar: Removing leading `/' from member names
2026-08-05T15:51:14.362094414Z tar: Removing leading `/' from member names
2026-08-05T15:51:14.364394811Z tar: Removing leading `/' from member names
2026-08-05T15:51:14.402969176Z tar: Removing leading `/' from member names
2026-08-05T15:51:14.463316438Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:14.535532389Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-08-05T15:51:14.564310694Z W0805 15:51:14.564260    1406 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-08-05T15:51:14.570859663Z Wrote inspect data to must-gather.
2026-08-05T15:51:14.706038258Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:14.711360493Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-08-05T15:51:14.726497415Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-08-05T15:51:14.842942031Z INFO: Waiting for network log collection to complete ...
2026-08-05T15:51:14.842989060Z INFO: Waiting for ovnk database copies to complete ...
2026-08-05T15:51:14.843508741Z INFO: Copying ovnk databases complete.
2026-08-05T15:51:14.845143111Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-08-05T15:51:14.847466956Z ovnk_database_store/
2026-08-05T15:51:14.847511534Z ovnk_database_store/ovnkube-node-4pzgd_sbdb
2026-08-05T15:51:14.866596660Z ovnk_database_store/ovnkube-node-prz9p_nbdb
2026-08-05T15:51:14.873835166Z ovnk_database_store/ovnkube-node-4pzgd_nbdb
2026-08-05T15:51:14.883241272Z ovnk_database_store/ovnkube-node-s9x49_sbdb
2026-08-05T15:51:14.900028258Z ovnk_database_store/ovnkube-node-prz9p_sbdb
2026-08-05T15:51:14.914087212Z Wrote inspect data to must-gather.
2026-08-05T15:51:14.920772772Z ovnk_database_store/ovnkube-node-s9x49_nbdb
2026-08-05T15:51:14.929795522Z INFO: Network log collection complete.
2026-08-05T15:51:14.930238083Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:14.945481284Z Gathering data for ns/openshift-network-console...
2026-08-05T15:51:14.951501208Z INFO: Getting status from alertmanager-main-0
2026-08-05T15:51:15.137067535Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:15.139362022Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-08-05T15:51:15.210576945Z Wrote inspect data to must-gather.
2026-08-05T15:51:15.343955809Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:15.410450563Z Gathering data for ns/openshift-console-operator...
2026-08-05T15:51:15.503668104Z Wrote inspect data to must-gather.
2026-08-05T15:51:15.540893452Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:15.542915724Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-08-05T15:51:15.737569196Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-08-05T15:51:15.741452708Z INFO: Fallback to identify the container image from release info
2026-08-05T15:51:15.750297518Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:15.802501605Z Gathering data for ns/openshift-console...
2026-08-05T15:51:15.940877925Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:15.943560464Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-08-05T15:51:16.082098005Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:16.274053266Z Wrote inspect data to /must-gather/istio.
2026-08-05T15:51:16.292189143Z Gathering data for ns/openshift-cluster-storage-operator...
2026-08-05T15:51:16.315810282Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-08-05T15:51:16.451026194Z daemonset.apps/perf-node-gather-daemonset created
2026-08-05T15:51:16.586239237Z Waiting for performance profile collector pods to become ready: 1
2026-08-05T15:51:16.827093615Z Gathering data for ns/openshift-dns-operator...
2026-08-05T15:51:16.947599565Z Done executing Istio gather script
2026-08-05T15:51:17.141866143Z Gathering data for ns/openshift-dns...
2026-08-05T15:51:17.704910660Z Waiting for performance profile collector pods to become ready: 2
2026-08-05T15:51:17.755225083Z Gathering data for ns/openshift-image-registry...
2026-08-05T15:51:17.856707570Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-05T15:51:18.195982082Z Gathering data for ns/openshift-ingress-operator...
2026-08-05T15:51:18.546436791Z Gathering data for ns/openshift-ingress...
2026-08-05T15:51:18.825432810Z Waiting for performance profile collector pods to become ready: 3
2026-08-05T15:51:18.865481076Z Gathering data for ns/openshift-ingress-canary...
2026-08-05T15:51:19.208679169Z Gathering data for ns/openshift-insights...
2026-08-05T15:51:19.924894572Z Waiting for performance profile collector pods to become ready: 4
2026-08-05T15:51:20.933045374Z Gathering data for ns/openshift-jobset-operator...
2026-08-05T15:51:21.032976260Z Waiting for performance profile collector pods to become ready: 5
2026-08-05T15:51:21.676946456Z Gathering data for ns/openshift-config...
2026-08-05T15:51:21.920302041Z Gathering data for ns/openshift-config-managed...
2026-08-05T15:51:22.136972450Z Waiting for performance profile collector pods to become ready: 6
2026-08-05T15:51:22.316486968Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-08-05T15:51:22.607802987Z Gathering data for ns/openshift-kube-apiserver...
2026-08-05T15:51:22.844048437Z Gathering data for ns/openshift-kube-controller-manager...
2026-08-05T15:51:22.862492284Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-05T15:51:23.130066423Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-08-05T15:51:23.247571599Z Waiting for performance profile collector pods to become ready: 7
2026-08-05T15:51:23.422261091Z Gathering data for ns/openshift-kube-scheduler...
2026-08-05T15:51:23.652325596Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-08-05T15:51:23.913600647Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-08-05T15:51:24.209293696Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-08-05T15:51:24.355711864Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-08-05T15:51:24.510342540Z Gathering data for ns/openshift-user-workload-monitoring...
2026-08-05T15:51:24.606275124Z Collecting performance related data for node ip-10-0-136-120.ec2.internal
2026-08-05T15:51:24.612741667Z Collecting performance related data for node ip-10-0-142-177.ec2.internal
2026-08-05T15:51:24.619450236Z Collecting performance related data for node ip-10-0-135-205.ec2.internal
2026-08-05T15:51:25.072813699Z Gathering data for ns/openshift-multus...
2026-08-05T15:51:26.536381864Z Gathering data for ns/openshift-ovn-kubernetes...
2026-08-05T15:51:27.792139274Z Gathering data for ns/openshift-host-network...
2026-08-05T15:51:27.867300758Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-05T15:51:28.073544482Z Gathering data for ns/openshift-network-diagnostics...
2026-08-05T15:51:28.588423197Z Gathering data for ns/openshift-network-node-identity...
2026-08-05T15:51:28.965233799Z Gathering data for ns/openshift-network-operator...
2026-08-05T15:51:29.327465298Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-08-05T15:51:29.590031854Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-08-05T15:51:30.104318462Z Gathering data for ns/openshift-apiserver-operator...
2026-08-05T15:51:30.393953834Z Gathering data for ns/openshift-apiserver...
2026-08-05T15:51:30.761149875Z Gathering data for ns/openshift-controller-manager-operator...
2026-08-05T15:51:31.011672279Z Gathering data for ns/openshift-controller-manager...
2026-08-05T15:51:31.283709558Z Gathering data for ns/openshift-cluster-samples-operator...
2026-08-05T15:51:31.917934693Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-08-05T15:51:32.223671038Z Gathering data for ns/openshift-service-ca-operator...
2026-08-05T15:51:32.280140307Z Collecting kubelet logs for node ip-10-0-136-120.ec2.internal
2026-08-05T15:51:32.281998632Z Collecting kubelet logs for node ip-10-0-142-177.ec2.internal
2026-08-05T15:51:32.284279939Z Collecting kubelet logs for node ip-10-0-135-205.ec2.internal
2026-08-05T15:51:32.584422953Z Gathering data for ns/openshift-service-ca...
2026-08-05T15:51:32.620194312Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vrz2f namespace
2026-08-05T15:51:32.626107045Z INFO: Node performance data collection complete.
2026-08-05T15:51:32.871708726Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-08-05T15:51:32.958972009Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-08-05T15:51:33.785285952Z Wrote inspect data to must-gather.
2026-08-05T15:51:33.785326022Z error: inspection completed with the errors occurred while gathering data:
2026-08-05T15:51:33.785326022Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-08-05T15:51:33.900017481Z error: the server doesn't have a resource type "clusters"
2026-08-05T15:51:34.030009341Z Caches written to disk