2026-04-17T14:56:42.294465668Z [disk usage checker] Started
2026-04-17T14:56:42.329092464Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T14:56:42.522741749Z Gathering data for ns/openshift-cluster-version...
2026-04-17T14:56:42.676478932Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T14:56:42.774311581Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T14:56:42.880838528Z Gathering data for ns/default...
2026-04-17T14:56:43.250584885Z Gathering data for ns/openshift...
2026-04-17T14:56:43.720030904Z Gathering data for ns/kube-system...
2026-04-17T14:56:44.608750616Z Gathering data for ns/openshift-etcd...
2026-04-17T14:56:44.734340182Z Waiting on subprocesses to finish execution.
2026-04-17T14:56:44.737902583Z INFO: Collecting host service logs for crio
2026-04-17T14:56:44.738291781Z INFO: Collecting host service logs for kubelet
2026-04-17T14:56:44.738636060Z INFO: Collecting host service logs for rpm-ostreed
2026-04-17T14:56:44.738953236Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-17T14:56:44.739302486Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-17T14:56:44.739634797Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-17T14:56:44.739979691Z INFO: Collecting host service logs for NetworkManager
2026-04-17T14:56:44.740385999Z INFO: Collecting host service logs for openvswitch
2026-04-17T14:56:44.740583751Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-17T14:56:44.740756245Z INFO: Collecting host service logs for ovs-configuration
2026-04-17T14:56:44.741102225Z INFO: Collecting host service logs for ovsdb-server
2026-04-17T14:56:44.741454915Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-17T14:56:44.741845348Z INFO: Waiting for worker host service log collection to complete ...
2026-04-17T14:56:44.745622827Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-17T14:56:44.872534637Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-17T14:56:44.891425145Z INFO: Gathering HAProxy config files
2026-04-17T14:56:45.158808459Z INFO: Waiting for node performance related collection to complete ...
2026-04-17T14:56:45.486068563Z INFO: "metallb-operator" not detected. Skipping.
2026-04-17T14:56:45.550826825Z No resources found
2026-04-17T14:56:45.570125160Z No resources found in openshift-etcd namespace.
2026-04-17T14:56:45.717813385Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-17T14:56:45.893185959Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-b4t8r
2026-04-17T14:56:45.898768510Z insights-runtime-extractor-56xpp
2026-04-17T14:56:45.898821704Z insights-runtime-extractor-hkr86
2026-04-17T14:56:45.898845528Z insights-runtime-extractor-zphcj
2026-04-17T14:56:45.961360181Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-17T14:56:45.989807549Z Wrote inspect data to must-gather.
2026-04-17T14:56:46.012667902Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-17T14:56:46.102449127Z Wrote inspect data to must-gather.
2026-04-17T14:56:46.102547104Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T14:56:46.102547104Z     namespaces "assisted-installer" not found
2026-04-17T14:56:46.282666478Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-17T14:56:46.302428907Z ERROR: No running kube-apiserver pods found
2026-04-17T14:56:46.345804358Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-17T14:56:46.390715635Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-17T14:56:46.390837794Z INFO: on-disk MachineConfig config collection complete.
2026-04-17T14:56:46.401839805Z error: the server doesn't have a resource type "performanceprofile"
2026-04-17T14:56:46.402118004Z INFO: Worker host service log collection to complete.
2026-04-17T14:56:46.592406547Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-17T14:56:46.592406547Z See 'oc rsync -h' for help and examples
2026-04-17T14:56:46.854410277Z No resources found
2026-04-17T14:56:46.935534799Z Wrote inspect data to must-gather.
2026-04-17T14:56:47.053994769Z INFO: 'previous-logs' folder not found on ip-10-0-132-119.ec2.internal, skipping...
2026-04-17T14:56:47.076103534Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T14:56:47.301705299Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-17T14:56:47.366107053Z Wrote inspect data to must-gather.
2026-04-17T14:56:47.391385277Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T14:56:47.478307076Z Wrote inspect data to must-gather.
2026-04-17T14:56:47.566539158Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-17T14:56:47.750446310Z Wrote inspect data to must-gather.
2026-04-17T14:56:47.810777262Z Wrote inspect data to must-gather.
2026-04-17T14:56:47.842064949Z INFO: 'previous-logs' folder not found on ip-10-0-138-3.ec2.internal, skipping...
2026-04-17T14:56:47.846630594Z Gathering data for ns/kuadrant-system...
2026-04-17T14:56:47.852586218Z INFO: Getting rules from prometheus-k8s-0
2026-04-17T14:56:47.922239805Z tar: Removing leading `/' from member names
2026-04-17T14:56:47.929253976Z INFO: HAProxy config collection complete.
2026-04-17T14:56:48.046931113Z error: the server doesn't have a resource type "machineconfigs"
2026-04-17T14:56:48.165065955Z Wrote inspect data to must-gather.
2026-04-17T14:56:48.190601328Z INFO: 'previous-logs' folder not found on ip-10-0-143-215.ec2.internal, skipping...
2026-04-17T14:56:48.190705936Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-17T14:56:48.190753780Z INFO: Machine Config Daemon termination log collection complete.
2026-04-17T14:56:48.322260008Z Wrote inspect data to must-gather.
2026-04-17T14:56:48.332751181Z INFO: Getting status/config from prometheus-k8s-0
2026-04-17T14:56:48.528543159Z Wrote inspect data to must-gather.
2026-04-17T14:56:48.633270280Z Wrote inspect data to must-gather.
2026-04-17T14:56:48.863324881Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-17T14:56:48.879425497Z Wrote inspect data to must-gather.
2026-04-17T14:56:48.901629646Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-17T14:56:49.060066669Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T14:56:49.188289786Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-17T14:56:49.262191807Z Wrote inspect data to must-gather.
2026-04-17T14:56:49.270744445Z INFO: INTERCONNECT MODE
2026-04-17T14:56:49.270779231Z INFO: Gathering ovn-kubernetes DBs
2026-04-17T14:56:49.463943436Z INFO: Gathering OVN_Northbound from ovnkube-node-5fm94...
2026-04-17T14:56:49.464270800Z INFO: Gathering OVN_Northbound from ovnkube-node-x4dft...
2026-04-17T14:56:49.464530220Z INFO: Gathering OVN_Northbound from ovnkube-node-zns76...
2026-04-17T14:56:49.464876350Z INFO: Gathering OVN_Southbound from ovnkube-node-5fm94...
2026-04-17T14:56:49.465139314Z INFO: Gathering OVN_Southbound from ovnkube-node-x4dft...
2026-04-17T14:56:49.465408505Z INFO: Gathering OVN_Southbound from ovnkube-node-zns76...
2026-04-17T14:56:49.480472473Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-17T14:56:49.635201126Z W0417 14:56:49.635134    1210 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-17T14:56:49.696947948Z Wrote inspect data to must-gather.
2026-04-17T14:56:49.757012533Z tar: Removing leading `/' from member names
2026-04-17T14:56:49.773569568Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T14:56:49.815294492Z tar: Removing leading `/' from member names
2026-04-17T14:56:49.889293063Z tar: Removing leading `/' from member names
2026-04-17T14:56:49.926330040Z tar: Removing leading `/' from member names
2026-04-17T14:56:49.930598063Z tar: Removing leading `/' from member names
2026-04-17T14:56:49.972754182Z tar: Removing leading `/' from member names
2026-04-17T14:56:50.032335990Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-17T14:56:50.115461212Z Gathering data for ns/openshift-monitoring...
2026-04-17T14:56:50.277452374Z Wrote inspect data to must-gather.
2026-04-17T14:56:50.545931181Z INFO: Getting status from alertmanager-main-0
2026-04-17T14:56:50.622010043Z Wrote inspect data to must-gather.
2026-04-17T14:56:50.644601229Z INFO: Waiting for network log collection to complete ...
2026-04-17T14:56:50.644784823Z INFO: Waiting for ovnk database copies to complete ...
2026-04-17T14:56:50.645418040Z INFO: Copying ovnk databases complete.
2026-04-17T14:56:50.647480143Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-04-17T14:56:50.649840816Z ovnk_database_store/
2026-04-17T14:56:50.649840816Z ovnk_database_store/ovnkube-node-5fm94_sbdb
2026-04-17T14:56:50.678234242Z ovnk_database_store/ovnkube-node-zns76_sbdb
2026-04-17T14:56:50.703707065Z ovnk_database_store/ovnkube-node-x4dft_nbdb
2026-04-17T14:56:50.713758132Z ovnk_database_store/ovnkube-node-zns76_nbdb
2026-04-17T14:56:50.723644045Z ovnk_database_store/ovnkube-node-x4dft_sbdb
2026-04-17T14:56:50.754474564Z ovnk_database_store/ovnkube-node-5fm94_nbdb
2026-04-17T14:56:50.767573337Z INFO: Network log collection complete.
2026-04-17T14:56:50.926491321Z Wrote inspect data to must-gather.
2026-04-17T14:56:51.212272087Z Wrote inspect data to must-gather.
2026-04-17T14:56:51.519433985Z Wrote inspect data to must-gather.
2026-04-17T14:56:51.816556679Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-17T14:56:51.821840912Z INFO: Fallback to identify the container image from release info
2026-04-17T14:56:52.295586451Z Gathering data for ns/openshift-network-console...
2026-04-17T14:56:52.385080801Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-17T14:56:52.404715533Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T14:56:52.527111601Z daemonset.apps/perf-node-gather-daemonset created
2026-04-17T14:56:52.695742151Z Waiting for performance profile collector pods to become ready: 1
2026-04-17T14:56:52.864340835Z Gathering data for ns/openshift-console-operator...
2026-04-17T14:56:53.368701936Z Gathering data for ns/openshift-console...
2026-04-17T14:56:53.635231180Z Wrote inspect data to must-gather.
2026-04-17T14:56:53.839653245Z Waiting for performance profile collector pods to become ready: 2
2026-04-17T14:56:53.926773747Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-17T14:56:54.364132322Z Gathering data for ns/openshift-dns-operator...
2026-04-17T14:56:54.703085176Z Gathering data for ns/openshift-dns...
2026-04-17T14:56:54.957235054Z Waiting for performance profile collector pods to become ready: 3
2026-04-17T14:56:55.317296852Z Gathering data for ns/openshift-image-registry...
2026-04-17T14:56:55.790072906Z Gathering data for ns/openshift-ingress-operator...
2026-04-17T14:56:56.082952340Z Waiting for performance profile collector pods to become ready: 4
2026-04-17T14:56:56.144188321Z Gathering data for ns/openshift-ingress...
2026-04-17T14:56:56.825159810Z Gathering data for ns/openshift-ingress-canary...
2026-04-17T14:56:57.211606802Z Waiting for performance profile collector pods to become ready: 5
2026-04-17T14:56:57.281645793Z Gathering data for ns/openshift-insights...
2026-04-17T14:56:57.411605734Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T14:56:58.318902746Z Waiting for performance profile collector pods to become ready: 6
2026-04-17T14:56:59.208047140Z Gathering data for ns/opendatahub...
2026-04-17T14:56:59.435734178Z Waiting for performance profile collector pods to become ready: 7
2026-04-17T14:57:00.518009617Z Gathering data for ns/openshift-lws-operator...
2026-04-17T14:57:00.545040823Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-17T14:57:00.787496301Z Collecting performance related data for node ip-10-0-132-119.ec2.internal
2026-04-17T14:57:00.794264314Z Collecting performance related data for node ip-10-0-138-3.ec2.internal
2026-04-17T14:57:00.801535607Z Collecting performance related data for node ip-10-0-143-215.ec2.internal
2026-04-17T14:57:02.019422974Z Gathering data for ns/openshift-config...
2026-04-17T14:57:02.355949339Z Gathering data for ns/openshift-config-managed...
2026-04-17T14:57:02.420419899Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T14:57:02.778691916Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-17T14:57:03.132146822Z Gathering data for ns/openshift-kube-apiserver...
2026-04-17T14:57:03.482956229Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-17T14:57:03.898496675Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-17T14:57:04.297797110Z Gathering data for ns/openshift-kube-scheduler...
2026-04-17T14:57:04.667602743Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-17T14:57:05.008583281Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-17T14:57:05.391017962Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-17T14:57:05.764021076Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-17T14:57:06.352582329Z Gathering data for ns/openshift-multus...
2026-04-17T14:57:07.425258582Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T14:57:07.845046629Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-17T14:57:09.022246032Z Collecting kubelet logs for node ip-10-0-132-119.ec2.internal
2026-04-17T14:57:09.024432128Z Collecting kubelet logs for node ip-10-0-138-3.ec2.internal
2026-04-17T14:57:09.026524795Z Collecting kubelet logs for node ip-10-0-143-215.ec2.internal
2026-04-17T14:57:09.402877948Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-17T14:57:09.410409878Z Gathering data for ns/openshift-host-network...
2026-04-17T14:57:09.412872222Z INFO: Node performance data collection complete.
2026-04-17T14:57:09.787850819Z Gathering data for ns/openshift-network-diagnostics...
2026-04-17T14:57:10.344635526Z Gathering data for ns/openshift-network-node-identity...
2026-04-17T14:57:10.852940520Z Gathering data for ns/openshift-network-operator...
2026-04-17T14:57:11.223548233Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-17T14:57:11.501451737Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-17T14:57:11.946400409Z Gathering data for ns/openshift-apiserver-operator...
2026-04-17T14:57:12.300397015Z Gathering data for ns/openshift-apiserver...
2026-04-17T14:57:12.429742647Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-04-17T14:57:12.752769835Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-17T14:57:13.084434251Z Gathering data for ns/openshift-controller-manager...
2026-04-17T14:57:13.402468729Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-17T14:57:14.108789416Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-17T14:57:14.431257169Z Gathering data for ns/openshift-service-ca-operator...
2026-04-17T14:57:14.772663926Z Gathering data for ns/openshift-service-ca...
2026-04-17T14:57:15.188820926Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-17T14:57:16.258128878Z Wrote inspect data to must-gather.
2026-04-17T14:57:16.258194686Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T14:57:16.258194686Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-17T14:57:16.393567730Z error: the server doesn't have a resource type "clusters"
2026-04-17T14:57:16.574522610Z Caches written to disk