2026-06-15T06:15:21.980325623Z [disk usage checker] Started
2026-06-15T06:15:21.983629205Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-15T06:15:22.184772025Z Gathering data for ns/openshift-cluster-version...
2026-06-15T06:15:22.376087147Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-15T06:15:22.460894245Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-15T06:15:22.548257299Z Gathering data for ns/default...
2026-06-15T06:15:22.869656285Z Gathering data for ns/openshift...
2026-06-15T06:15:23.286073418Z Gathering data for ns/kube-system...
2026-06-15T06:15:23.981663240Z Gathering data for ns/openshift-etcd...
2026-06-15T06:15:24.295503536Z Wrote inspect data to must-gather.
2026-06-15T06:15:24.295535095Z error: inspection completed with the errors occurred while gathering data:
2026-06-15T06:15:24.295535095Z     namespaces "assisted-installer" not found
2026-06-15T06:15:24.848355103Z Waiting on subprocesses to finish execution.
2026-06-15T06:15:24.862267968Z Executing Istio gather script
2026-06-15T06:15:24.864050377Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-15T06:15:24.869366814Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-15T06:15:24.871586332Z INFO: Gathering HAProxy config files
2026-06-15T06:15:24.875878175Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-15T06:15:24.887404036Z INFO: Collecting host service logs for crio
2026-06-15T06:15:24.887845946Z INFO: Collecting host service logs for kubelet
2026-06-15T06:15:24.888256540Z INFO: Collecting host service logs for rpm-ostreed
2026-06-15T06:15:24.889648873Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-15T06:15:24.890017247Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-15T06:15:24.890185817Z INFO: Waiting for node performance related collection to complete ...
2026-06-15T06:15:24.890342510Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-15T06:15:24.890638759Z INFO: Collecting host service logs for NetworkManager
2026-06-15T06:15:24.890968331Z INFO: Collecting host service logs for openvswitch
2026-06-15T06:15:24.891264866Z INFO: Collecting host service logs for ovs-configuration
2026-06-15T06:15:24.891584221Z INFO: Collecting host service logs for ovsdb-server
2026-06-15T06:15:24.891956504Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-15T06:15:24.892317521Z INFO: Waiting for worker host service log collection to complete ...
2026-06-15T06:15:25.363421237Z No resources found
2026-06-15T06:15:25.372856245Z INFO: Collecting Insights Archives from insights-operator-5bbd86d6bd-fh9nd
2026-06-15T06:15:25.373098979Z insights-runtime-extractor-g7xhw
2026-06-15T06:15:25.373122782Z insights-runtime-extractor-jb6wq
2026-06-15T06:15:25.373143195Z insights-runtime-extractor-tzbhd
2026-06-15T06:15:25.683148293Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-15T06:15:25.683148293Z See 'oc rsync -h' for help and examples
2026-06-15T06:15:25.685572423Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-15T06:15:25.685572423Z INFO: on-disk MachineConfig config collection complete.
2026-06-15T06:15:25.740197940Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-15T06:15:26.252444956Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-15T06:15:26.365274734Z error: the server doesn't have a resource type "performanceprofile"
2026-06-15T06:15:26.671948954Z Wrote inspect data to must-gather.
2026-06-15T06:15:26.721056878Z ERROR: No running kube-apiserver pods found
2026-06-15T06:15:26.814104506Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-15T06:15:26.855887602Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-15T06:15:26.915953307Z INFO: "metallb-operator" not detected. Skipping.
2026-06-15T06:15:27.010705388Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-15T06:15:27.113859123Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-15T06:15:27.138074941Z No resources found
2026-06-15T06:15:27.172666934Z INFO: Worker host service log collection to complete.
2026-06-15T06:15:27.187906856Z Wrote inspect data to must-gather.
2026-06-15T06:15:27.193182559Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-15T06:15:27.222231240Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-15T06:15:27.233022943Z No resources found in openshift-etcd namespace.
2026-06-15T06:15:27.293525257Z Wrote inspect data to must-gather.
2026-06-15T06:15:27.319915215Z INFO: 'previous-logs' folder not found on ip-10-0-135-167.ec2.internal, skipping...
2026-06-15T06:15:27.411412072Z Wrote inspect data to must-gather.
2026-06-15T06:15:27.416886861Z Wrote inspect data to must-gather.
2026-06-15T06:15:27.509447907Z Wrote inspect data to must-gather.
2026-06-15T06:15:27.583207739Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-15T06:15:27.606367295Z tar: Removing leading `/' from member names
2026-06-15T06:15:27.609917769Z Wrote inspect data to must-gather.
2026-06-15T06:15:27.610882195Z INFO: HAProxy config collection complete.
2026-06-15T06:15:27.792961034Z Wrote inspect data to must-gather.
2026-06-15T06:15:27.807988007Z INFO: 'previous-logs' folder not found on ip-10-0-136-138.ec2.internal, skipping...
2026-06-15T06:15:27.838093316Z error: the server doesn't have a resource type "machineconfigs"
2026-06-15T06:15:27.999145044Z Gathering data for ns/openshift-monitoring...
2026-06-15T06:15:28.069288004Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-15T06:15:28.080377621Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-15T06:15:28.095405987Z INFO: Getting rules from prometheus-k8s-0
2026-06-15T06:15:28.144815758Z Wrote inspect data to must-gather.
2026-06-15T06:15:28.150172100Z INFO: 'previous-logs' folder not found on ip-10-0-140-64.ec2.internal, skipping...
2026-06-15T06:15:28.150223081Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-15T06:15:28.150254685Z INFO: Machine Config Daemon termination log collection complete.
2026-06-15T06:15:28.316387651Z Wrote inspect data to must-gather.
2026-06-15T06:15:28.411215195Z Wrote inspect data to must-gather.
2026-06-15T06:15:28.491994809Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-15T06:15:28.503423960Z INFO: Getting status/config from prometheus-k8s-0
2026-06-15T06:15:28.730974413Z Wrote inspect data to must-gather.
2026-06-15T06:15:28.733114438Z INFO: INTERCONNECT MODE
2026-06-15T06:15:28.733139439Z INFO: Gathering ovn-kubernetes DBs
2026-06-15T06:15:28.913622141Z INFO: Gathering OVN_Northbound from ovnkube-node-bcprp...
2026-06-15T06:15:28.913902729Z INFO: Gathering OVN_Northbound from ovnkube-node-gnn2f...
2026-06-15T06:15:28.914153204Z INFO: Gathering OVN_Northbound from ovnkube-node-lk9qv...
2026-06-15T06:15:28.914479554Z INFO: Gathering OVN_Southbound from ovnkube-node-bcprp...
2026-06-15T06:15:28.914778914Z INFO: Gathering OVN_Southbound from ovnkube-node-gnn2f...
2026-06-15T06:15:28.915048776Z INFO: Gathering OVN_Southbound from ovnkube-node-lk9qv...
2026-06-15T06:15:28.951826441Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-15T06:15:29.023244056Z Wrote inspect data to must-gather.
2026-06-15T06:15:29.238956689Z tar: Removing leading `/' from member names
2026-06-15T06:15:29.247191967Z tar: Removing leading `/' from member names
2026-06-15T06:15:29.264795447Z tar: Removing leading `/' from member names
2026-06-15T06:15:29.272842489Z tar: Removing leading `/' from member names
2026-06-15T06:15:29.280201726Z tar: Removing leading `/' from member names
2026-06-15T06:15:29.321997775Z tar: Removing leading `/' from member names
2026-06-15T06:15:29.333113790Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-15T06:15:29.416365097Z Wrote inspect data to must-gather.
2026-06-15T06:15:29.549866623Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-15T06:15:29.554067195Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-15T06:15:29.616703141Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-15T06:15:29.663065049Z Wrote inspect data to must-gather.
2026-06-15T06:15:29.781099255Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:29.816445750Z INFO: Waiting for network log collection to complete ...
2026-06-15T06:15:29.816515891Z INFO: Waiting for ovnk database copies to complete ...
2026-06-15T06:15:29.817022143Z INFO: Copying ovnk databases complete.
2026-06-15T06:15:29.818613800Z 3.4M	must-gather/network_logs/ovnk_database_store
2026-06-15T06:15:29.821185372Z ovnk_database_store/
2026-06-15T06:15:29.821185372Z ovnk_database_store/ovnkube-node-gnn2f_nbdb
2026-06-15T06:15:29.830995526Z ovnk_database_store/ovnkube-node-bcprp_sbdb
2026-06-15T06:15:29.852151102Z ovnk_database_store/ovnkube-node-lk9qv_sbdb
2026-06-15T06:15:29.873685006Z ovnk_database_store/ovnkube-node-bcprp_nbdb
2026-06-15T06:15:29.882516928Z ovnk_database_store/ovnkube-node-gnn2f_sbdb
2026-06-15T06:15:29.902719809Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-15T06:15:29.903401470Z Wrote inspect data to must-gather.
2026-06-15T06:15:29.904569676Z ovnk_database_store/ovnkube-node-lk9qv_nbdb
2026-06-15T06:15:29.916785184Z INFO: Network log collection complete.
2026-06-15T06:15:30.028257078Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:30.031653446Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-15T06:15:30.127637096Z W0615 06:15:30.127589    1483 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-15T06:15:30.132497604Z Wrote inspect data to must-gather.
2026-06-15T06:15:30.213077328Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-15T06:15:30.267499761Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:30.323625094Z INFO: Getting status from alertmanager-main-0
2026-06-15T06:15:30.435371213Z Gathering data for ns/openshift-network-console...
2026-06-15T06:15:30.480347401Z Wrote inspect data to must-gather.
2026-06-15T06:15:30.519429051Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:30.523828348Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-15T06:15:30.731165539Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:30.792922419Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-15T06:15:30.795465486Z INFO: Fallback to identify the container image from release info
2026-06-15T06:15:30.907101401Z Gathering data for ns/openshift-console-operator...
2026-06-15T06:15:30.953489538Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:30.956754890Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-15T06:15:31.183608713Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:31.353383944Z Gathering data for ns/openshift-console...
2026-06-15T06:15:31.393148797Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:31.395825411Z Inspecting resource crd/inferencemodelrewrites.inference.networking.x-k8s.io
2026-06-15T06:15:31.539316590Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d66a6b2ec52dede5d83ce04f6abfc519df1fd7dc36248741e03ecb9bf8d1762
2026-06-15T06:15:31.590755824Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:31.687179965Z daemonset.apps/perf-node-gather-daemonset created
2026-06-15T06:15:31.826525455Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:31.829807367Z Inspecting resource crd/inferenceobjectives.inference.networking.x-k8s.io
2026-06-15T06:15:31.847308613Z Waiting for performance profile collector pods to become ready: 1
2026-06-15T06:15:31.921798955Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-15T06:15:32.019324984Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-15T06:15:32.031814426Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:32.261973655Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:32.265089772Z Inspecting resource crd/inferencepoolimports.inference.networking.x-k8s.io
2026-06-15T06:15:32.454973333Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:32.475270295Z Gathering data for ns/openshift-dns-operator...
2026-06-15T06:15:32.666311186Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:32.669640673Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-06-15T06:15:32.781435853Z Gathering data for ns/openshift-dns...
2026-06-15T06:15:32.845005985Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:32.978071799Z Waiting for performance profile collector pods to become ready: 2
2026-06-15T06:15:33.049216911Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:33.053625200Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-06-15T06:15:33.231220301Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:33.415616710Z Gathering data for ns/openshift-image-registry...
2026-06-15T06:15:33.440465527Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:33.443521647Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-15T06:15:33.623029328Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:33.816902340Z Wrote inspect data to /must-gather/istio.
2026-06-15T06:15:33.911970786Z Gathering data for ns/openshift-ingress-operator...
2026-06-15T06:15:34.133030195Z Waiting for performance profile collector pods to become ready: 3
2026-06-15T06:15:34.356716259Z Gathering data for ns/openshift-ingress...
2026-06-15T06:15:34.580081044Z Done executing Istio gather script
2026-06-15T06:15:34.703880511Z Gathering data for ns/openshift-ingress-canary...
2026-06-15T06:15:35.095155849Z Gathering data for ns/openshift-insights...
2026-06-15T06:15:35.248064863Z Waiting for performance profile collector pods to become ready: 4
2026-06-15T06:15:36.373214079Z Waiting for performance profile collector pods to become ready: 5
2026-06-15T06:15:37.024554360Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-15T06:15:37.254008357Z Gathering data for ns/kserve...
2026-06-15T06:15:37.497758362Z Waiting for performance profile collector pods to become ready: 6
2026-06-15T06:15:38.618806348Z Waiting for performance profile collector pods to become ready: 7
2026-06-15T06:15:38.843968587Z Gathering data for ns/openshift-config...
2026-06-15T06:15:39.122321053Z Gathering data for ns/openshift-config-managed...
2026-06-15T06:15:39.558027366Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-15T06:15:39.738771562Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-15T06:15:39.875133954Z Gathering data for ns/openshift-kube-apiserver...
2026-06-15T06:15:40.026308896Z Collecting performance related data for node ip-10-0-135-167.ec2.internal
2026-06-15T06:15:40.033411080Z Collecting performance related data for node ip-10-0-136-138.ec2.internal
2026-06-15T06:15:40.042261697Z Collecting performance related data for node ip-10-0-140-64.ec2.internal
2026-06-15T06:15:40.205490890Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-15T06:15:40.538753243Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-15T06:15:40.877383580Z Gathering data for ns/openshift-kube-scheduler...
2026-06-15T06:15:41.219152517Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-15T06:15:41.557003976Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-15T06:15:41.897314470Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-15T06:15:42.029151964Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-15T06:15:42.205405266Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-15T06:15:42.760479007Z Gathering data for ns/openshift-multus...
2026-06-15T06:15:44.152241144Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-15T06:15:45.395290895Z Gathering data for ns/openshift-host-network...
2026-06-15T06:15:45.744325691Z Gathering data for ns/openshift-network-diagnostics...
2026-06-15T06:15:46.280712002Z Gathering data for ns/openshift-network-node-identity...
2026-06-15T06:15:46.701342814Z Gathering data for ns/openshift-network-operator...
2026-06-15T06:15:47.033743504Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-06-15T06:15:47.071613175Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-15T06:15:47.351213786Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-15T06:15:47.838504564Z Gathering data for ns/openshift-apiserver-operator...
2026-06-15T06:15:48.134557820Z Gathering data for ns/openshift-apiserver...
2026-06-15T06:15:48.331928761Z Collecting kubelet logs for node ip-10-0-135-167.ec2.internal
2026-06-15T06:15:48.334271668Z Collecting kubelet logs for node ip-10-0-136-138.ec2.internal
2026-06-15T06:15:48.336458956Z Collecting kubelet logs for node ip-10-0-140-64.ec2.internal
2026-06-15T06:15:48.575314701Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-15T06:15:48.743407429Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-zskbr namespace
2026-06-15T06:15:48.749085447Z INFO: Node performance data collection complete.
2026-06-15T06:15:48.886920987Z Gathering data for ns/openshift-controller-manager...
2026-06-15T06:15:49.161199358Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-15T06:15:49.796774955Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-15T06:15:50.067796884Z Gathering data for ns/openshift-service-ca-operator...
2026-06-15T06:15:50.356342593Z Gathering data for ns/openshift-service-ca...
2026-06-15T06:15:50.711579660Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-15T06:15:51.593477227Z Wrote inspect data to must-gather.
2026-06-15T06:15:51.593527361Z error: inspection completed with the errors occurred while gathering data:
2026-06-15T06:15:51.593527361Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-15T06:15:51.729844652Z error: the server doesn't have a resource type "clusters"
2026-06-15T06:15:51.872175551Z Caches written to disk