2026-09-17T16:35:10.077311292Z [disk usage checker] Started
2026-09-17T16:35:10.080945245Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T16:35:10.463233754Z Gathering data for ns/openshift-cluster-version...
2026-09-17T16:35:10.618581235Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T16:35:10.732960900Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:35:10.837748676Z Gathering data for ns/default...
2026-09-17T16:35:11.179887166Z Gathering data for ns/openshift...
2026-09-17T16:35:11.636355059Z Gathering data for ns/kube-system...
2026-09-17T16:35:12.527009031Z Gathering data for ns/openshift-etcd...
2026-09-17T16:35:12.897088489Z Wrote inspect data to must-gather.
2026-09-17T16:35:12.897195789Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T16:35:12.897195789Z     namespaces "assisted-installer" not found
2026-09-17T16:35:13.180233581Z Waiting on subprocesses to finish execution.
2026-09-17T16:35:13.187271237Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-09-17T16:35:13.190438806Z Executing Istio gather script
2026-09-17T16:35:13.199072121Z INFO: Gathering HAProxy config files
2026-09-17T16:35:13.201615482Z INFO: Gathering machine config daemon's old logs from all nodes
2026-09-17T16:35:13.216962310Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-09-17T16:35:13.221698474Z INFO: Collecting host service logs for crio
2026-09-17T16:35:13.222082729Z INFO: Collecting host service logs for kubelet
2026-09-17T16:35:13.222452456Z INFO: Collecting host service logs for rpm-ostreed
2026-09-17T16:35:13.222831088Z INFO: Collecting host service logs for ostree-finalize-staged
2026-09-17T16:35:13.223260422Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-09-17T16:35:13.223655768Z INFO: Collecting host service logs for machine-config-daemon-host
2026-09-17T16:35:13.224041318Z INFO: Collecting host service logs for NetworkManager
2026-09-17T16:35:13.224451798Z INFO: Collecting host service logs for openvswitch
2026-09-17T16:35:13.225017672Z INFO: Collecting host service logs for ovs-configuration
2026-09-17T16:35:13.225386361Z INFO: Collecting host service logs for ovsdb-server
2026-09-17T16:35:13.225765486Z INFO: Collecting host service logs for ovs-vswitchd
2026-09-17T16:35:13.226324420Z INFO: Waiting for worker host service log collection to complete ...
2026-09-17T16:35:13.316647605Z INFO: Waiting for node performance related collection to complete ...
2026-09-17T16:35:13.719989529Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-09-17T16:35:13.788779637Z INFO: Collecting Insights Archives from insights-operator-5844b856db-9jg52
2026-09-17T16:35:13.788779637Z insights-runtime-extractor-fxkqg
2026-09-17T16:35:13.788779637Z insights-runtime-extractor-vfpmt
2026-09-17T16:35:13.788779637Z insights-runtime-extractor-wn4ng
2026-09-17T16:35:13.806781206Z No resources found
2026-09-17T16:35:13.870174462Z INFO: "metallb-operator" not detected. Skipping.
2026-09-17T16:35:14.021397900Z No resources found in openshift-etcd namespace.
2026-09-17T16:35:14.195966423Z INFO: Found 1 replicas - prometheus-k8s-0
2026-09-17T16:35:14.317019730Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-09-17T16:35:14.359673354Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-09-17T16:35:14.359673354Z See 'oc rsync -h' for help and examples
2026-09-17T16:35:14.636181315Z INFO: "sriov-network-operator" not detected. Skipping.
2026-09-17T16:35:14.767720735Z INFO: Worker host service log collection to complete.
2026-09-17T16:35:14.954199541Z error: the server doesn't have a resource type "performanceprofile"
2026-09-17T16:35:14.989748360Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-09-17T16:35:14.989779759Z INFO: on-disk MachineConfig config collection complete.
2026-09-17T16:35:15.049072047Z Wrote inspect data to must-gather.
2026-09-17T16:35:15.142182168Z Wrote inspect data to must-gather.
2026-09-17T16:35:15.162891988Z ERROR: No running kube-apiserver pods found
2026-09-17T16:35:15.175252803Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T16:35:15.211528495Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-09-17T16:35:15.334549845Z INFO: 'previous-logs' folder not found on ip-10-0-128-144.ec2.internal, skipping...
2026-09-17T16:35:15.382349464Z No resources found
2026-09-17T16:35:15.490765499Z INFO: Waiting for HAProxy config collection to complete ...
2026-09-17T16:35:15.532095158Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-09-17T16:35:15.554743061Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:35:15.814562735Z Wrote inspect data to must-gather.
2026-09-17T16:35:15.818048979Z Wrote inspect data to must-gather.
2026-09-17T16:35:15.824670181Z Wrote inspect data to must-gather.
2026-09-17T16:35:15.838498183Z Wrote inspect data to must-gather.
2026-09-17T16:35:15.949873160Z Wrote inspect data to must-gather.
2026-09-17T16:35:16.008164397Z Gathering data for ns/openshift-monitoring...
2026-09-17T16:35:16.043028506Z tar: Removing leading `/' from member names
2026-09-17T16:35:16.054945180Z INFO: HAProxy config collection complete.
2026-09-17T16:35:16.116528394Z error: the server doesn't have a resource type "machineconfigs"
2026-09-17T16:35:16.124536830Z INFO: 'previous-logs' folder not found on ip-10-0-140-157.ec2.internal, skipping...
2026-09-17T16:35:16.187628991Z Wrote inspect data to must-gather.
2026-09-17T16:35:16.279030743Z INFO: Getting rules from prometheus-k8s-0
2026-09-17T16:35:16.301978365Z error: the server doesn't have a resource type "machineconfigpools"
2026-09-17T16:35:16.435011894Z INFO: 'previous-logs' folder not found on ip-10-0-143-133.ec2.internal, skipping...
2026-09-17T16:35:16.435047351Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-09-17T16:35:16.435075311Z INFO: Machine Config Daemon termination log collection complete.
2026-09-17T16:35:16.507029983Z Wrote inspect data to must-gather.
2026-09-17T16:35:16.577420109Z Wrote inspect data to must-gather.
2026-09-17T16:35:16.772004748Z INFO: Getting status/config from prometheus-k8s-0
2026-09-17T16:35:16.780602903Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-09-17T16:35:16.792028601Z error: the server doesn't have a resource type "kubeletconfigs"
2026-09-17T16:35:16.810288300Z Wrote inspect data to must-gather.
2026-09-17T16:35:17.030687688Z Wrote inspect data to must-gather.
2026-09-17T16:35:17.121459374Z INFO: INTERCONNECT MODE
2026-09-17T16:35:17.121647480Z INFO: Gathering ovn-kubernetes DBs
2026-09-17T16:35:17.213396071Z INFO: Getting status/flags from prometheus-k8s-0
2026-09-17T16:35:17.280938944Z Wrote inspect data to must-gather.
2026-09-17T16:35:17.300744059Z INFO: Gathering OVN_Northbound from ovnkube-node-486bt...
2026-09-17T16:35:17.301064026Z INFO: Gathering OVN_Northbound from ovnkube-node-z7swp...
2026-09-17T16:35:17.301280433Z INFO: Gathering OVN_Northbound from ovnkube-node-zpzbl...
2026-09-17T16:35:17.301579303Z INFO: Gathering OVN_Southbound from ovnkube-node-486bt...
2026-09-17T16:35:17.301933497Z INFO: Gathering OVN_Southbound from ovnkube-node-z7swp...
2026-09-17T16:35:17.302252018Z INFO: Gathering OVN_Southbound from ovnkube-node-zpzbl...
2026-09-17T16:35:17.568335993Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-09-17T16:35:17.631990502Z tar: Removing leading `/' from member names
2026-09-17T16:35:17.634282493Z tar: Removing leading `/' from member names
2026-09-17T16:35:17.641341815Z tar: Removing leading `/' from member names
2026-09-17T16:35:17.643628452Z Wrote inspect data to must-gather.
2026-09-17T16:35:17.651416518Z tar: Removing leading `/' from member names
2026-09-17T16:35:17.659178805Z tar: Removing leading `/' from member names
2026-09-17T16:35:17.688977855Z tar: Removing leading `/' from member names
2026-09-17T16:35:17.884319163Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-09-17T16:35:17.907692342Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-09-17T16:35:17.942376899Z Wrote inspect data to must-gather.
2026-09-17T16:35:17.969057150Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-09-17T16:35:18.195006422Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-09-17T16:35:18.195574355Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:18.211097999Z Wrote inspect data to must-gather.
2026-09-17T16:35:18.232439484Z INFO: Waiting for network log collection to complete ...
2026-09-17T16:35:18.232505487Z INFO: Waiting for ovnk database copies to complete ...
2026-09-17T16:35:18.232980281Z INFO: Copying ovnk databases complete.
2026-09-17T16:35:18.235149400Z 4.8M	must-gather/network_logs/ovnk_database_store
2026-09-17T16:35:18.238075510Z ovnk_database_store/
2026-09-17T16:35:18.238075510Z ovnk_database_store/ovnkube-node-z7swp_nbdb
2026-09-17T16:35:18.249736029Z ovnk_database_store/ovnkube-node-z7swp_sbdb
2026-09-17T16:35:18.278898495Z ovnk_database_store/ovnkube-node-486bt_nbdb
2026-09-17T16:35:18.292886809Z ovnk_database_store/ovnkube-node-zpzbl_sbdb
2026-09-17T16:35:18.324642079Z ovnk_database_store/ovnkube-node-zpzbl_nbdb
2026-09-17T16:35:18.337736114Z ovnk_database_store/ovnkube-node-486bt_sbdb
2026-09-17T16:35:18.376130741Z INFO: Network log collection complete.
2026-09-17T16:35:18.445555073Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:18.450516803Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-09-17T16:35:18.453105709Z W0917 16:35:18.453066    1515 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-09-17T16:35:18.458153505Z Wrote inspect data to must-gather.
2026-09-17T16:35:18.564616577Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-09-17T16:35:18.664815237Z INFO: Getting status from alertmanager-main-0
2026-09-17T16:35:18.698726984Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:18.790217720Z Gathering data for ns/openshift-network-console...
2026-09-17T16:35:18.791281925Z Wrote inspect data to must-gather.
2026-09-17T16:35:18.941153117Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:18.945174618Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-09-17T16:35:19.096894948Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-09-17T16:35:19.100644458Z INFO: Fallback to identify the container image from release info
2026-09-17T16:35:19.162741896Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:19.344261341Z Gathering data for ns/openshift-console-operator...
2026-09-17T16:35:19.380093739Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:19.383044572Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-09-17T16:35:19.619102699Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:19.714074929Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:667babb345aeb39d2681913fd91a5c163d48d4ab8a3ba0d3156502181ea1177a
2026-09-17T16:35:19.793543427Z Gathering data for ns/openshift-console...
2026-09-17T16:35:19.854873797Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:19.858789637Z Inspecting resource crd/inferencepools.inference.networking.k8s.io
2026-09-17T16:35:19.859585171Z daemonset.apps/perf-node-gather-daemonset created
2026-09-17T16:35:20.017961650Z Waiting for performance profile collector pods to become ready: 1
2026-09-17T16:35:20.054991533Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:20.181659413Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T16:35:20.252890766Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:20.257703382Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io
2026-09-17T16:35:20.463089675Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:20.513409820Z Gathering data for ns/openshift-cluster-storage-operator...
2026-09-17T16:35:20.677106588Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:20.680671269Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-09-17T16:35:20.858198364Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:21.087278591Z Wrote inspect data to /must-gather/istio.
2026-09-17T16:35:21.098486446Z Gathering data for ns/openshift-dns-operator...
2026-09-17T16:35:21.189458496Z Waiting for performance profile collector pods to become ready: 2
2026-09-17T16:35:21.458416857Z Gathering data for ns/openshift-dns...
2026-09-17T16:35:21.800181950Z Done executing Istio gather script
2026-09-17T16:35:22.113189233Z Gathering data for ns/openshift-image-registry...
2026-09-17T16:35:22.328961118Z Waiting for performance profile collector pods to become ready: 3
2026-09-17T16:35:22.656664783Z Gathering data for ns/openshift-ingress-operator...
2026-09-17T16:35:23.034060678Z Gathering data for ns/openshift-ingress...
2026-09-17T16:35:23.450509876Z Gathering data for ns/openshift-ingress-canary...
2026-09-17T16:35:23.464571340Z Waiting for performance profile collector pods to become ready: 4
2026-09-17T16:35:23.976903552Z Gathering data for ns/openshift-insights...
2026-09-17T16:35:24.581325552Z Waiting for performance profile collector pods to become ready: 5
2026-09-17T16:35:25.186613794Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T16:35:25.705967509Z Waiting for performance profile collector pods to become ready: 6
2026-09-17T16:35:26.709127618Z Gathering data for ns/kserve...
2026-09-17T16:35:26.838952165Z Waiting for performance profile collector pods to become ready: 7
2026-09-17T16:35:27.982303889Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-09-17T16:35:28.258208899Z Collecting performance related data for node ip-10-0-143-133.ec2.internal
2026-09-17T16:35:28.275994890Z Collecting performance related data for node ip-10-0-140-157.ec2.internal
2026-09-17T16:35:28.283896538Z Collecting performance related data for node ip-10-0-128-144.ec2.internal
2026-09-17T16:35:29.128146618Z Gathering data for ns/openshift-config...
2026-09-17T16:35:29.527369993Z Gathering data for ns/openshift-config-managed...
2026-09-17T16:35:30.042566882Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-09-17T16:35:30.191475872Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T16:35:30.335790624Z Gathering data for ns/openshift-kube-apiserver...
2026-09-17T16:35:30.681134847Z Gathering data for ns/openshift-kube-controller-manager...
2026-09-17T16:35:31.086551005Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-09-17T16:35:31.442114883Z Gathering data for ns/openshift-kube-scheduler...
2026-09-17T16:35:31.764488693Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-09-17T16:35:32.154188131Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-09-17T16:35:32.537065631Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-09-17T16:35:32.926568790Z Gathering data for ns/openshift-user-workload-monitoring...
2026-09-17T16:35:33.537069564Z Gathering data for ns/openshift-multus...
2026-09-17T16:35:35.197276428Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T16:35:35.380134054Z Gathering data for ns/openshift-ovn-kubernetes...
2026-09-17T16:35:36.977614040Z Collecting kubelet logs for node ip-10-0-143-133.ec2.internal
2026-09-17T16:35:36.979756856Z Collecting kubelet logs for node ip-10-0-140-157.ec2.internal
2026-09-17T16:35:36.983298061Z Collecting kubelet logs for node ip-10-0-128-144.ec2.internal
2026-09-17T16:35:37.075607016Z Gathering data for ns/openshift-host-network...
2026-09-17T16:35:37.420950312Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-bp66r namespace
2026-09-17T16:35:37.427413974Z INFO: Node performance data collection complete.
2026-09-17T16:35:37.470699077Z Gathering data for ns/openshift-network-diagnostics...
2026-09-17T16:35:38.060340479Z Gathering data for ns/openshift-network-node-identity...
2026-09-17T16:35:38.529269213Z Gathering data for ns/openshift-network-operator...
2026-09-17T16:35:39.084818689Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-09-17T16:35:39.476580687Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-09-17T16:35:40.117995725Z Gathering data for ns/openshift-apiserver-operator...
2026-09-17T16:35:40.202876513Z [disk usage checker] Volume usage percentage: current = 35 ; allowed = 70
2026-09-17T16:35:40.510282837Z Gathering data for ns/openshift-apiserver...
2026-09-17T16:35:40.944405920Z Gathering data for ns/openshift-controller-manager-operator...
2026-09-17T16:35:41.280683410Z Gathering data for ns/openshift-controller-manager...
2026-09-17T16:35:41.615653364Z Gathering data for ns/openshift-cluster-samples-operator...
2026-09-17T16:35:42.347001387Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-09-17T16:35:42.678422709Z Gathering data for ns/openshift-service-ca-operator...
2026-09-17T16:35:43.043191971Z Gathering data for ns/openshift-service-ca...
2026-09-17T16:35:43.449493095Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-09-17T16:35:44.646838208Z Wrote inspect data to must-gather.
2026-09-17T16:35:44.646886119Z error: inspection completed with the errors occurred while gathering data:
2026-09-17T16:35:44.646886119Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-09-17T16:35:44.786391895Z error: the server doesn't have a resource type "clusters"
2026-09-17T16:35:44.938089549Z Caches written to disk