2026-04-17T18:12:38.647116552Z [disk usage checker] Started
2026-04-17T18:12:38.680422678Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T18:12:38.860429237Z Gathering data for ns/openshift-cluster-version...
2026-04-17T18:12:38.998690417Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T18:12:39.085745218Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T18:12:39.167537090Z Gathering data for ns/default...
2026-04-17T18:12:39.436052882Z Gathering data for ns/openshift...
2026-04-17T18:12:39.850866032Z Gathering data for ns/kube-system...
2026-04-17T18:12:40.534213176Z Gathering data for ns/openshift-etcd...
2026-04-17T18:12:40.842992864Z Wrote inspect data to must-gather.
2026-04-17T18:12:40.843025928Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T18:12:40.843025928Z     namespaces "assisted-installer" not found
2026-04-17T18:12:41.131878552Z Waiting on subprocesses to finish execution.
2026-04-17T18:12:41.138642220Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-17T18:12:41.147351120Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-17T18:12:41.148639281Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-17T18:12:41.155530709Z INFO: Gathering HAProxy config files
2026-04-17T18:12:41.170697470Z INFO: Collecting host service logs for crio
2026-04-17T18:12:41.171071268Z INFO: Collecting host service logs for kubelet
2026-04-17T18:12:41.171430292Z INFO: Collecting host service logs for rpm-ostreed
2026-04-17T18:12:41.171776385Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-17T18:12:41.172127089Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-17T18:12:41.172470746Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-17T18:12:41.172824263Z INFO: Collecting host service logs for NetworkManager
2026-04-17T18:12:41.173159177Z INFO: Collecting host service logs for openvswitch
2026-04-17T18:12:41.173483564Z INFO: Collecting host service logs for ovs-configuration
2026-04-17T18:12:41.173842794Z INFO: Collecting host service logs for ovsdb-server
2026-04-17T18:12:41.175989373Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-17T18:12:41.176380996Z INFO: Waiting for worker host service log collection to complete ...
2026-04-17T18:12:41.238820950Z INFO: Waiting for node performance related collection to complete ...
2026-04-17T18:12:41.742624558Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-9dmmr
2026-04-17T18:12:41.742686655Z insights-runtime-extractor-88mpn
2026-04-17T18:12:41.742708345Z insights-runtime-extractor-d966q
2026-04-17T18:12:41.742729858Z insights-runtime-extractor-js8tl
2026-04-17T18:12:41.802486849Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-17T18:12:41.902861711Z INFO: "metallb-operator" not detected. Skipping.
2026-04-17T18:12:41.914946300Z Wrote inspect data to must-gather.
2026-04-17T18:12:42.007912533Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-17T18:12:42.013738571Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-17T18:12:42.013820855Z INFO: on-disk MachineConfig config collection complete.
2026-04-17T18:12:42.090665792Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-17T18:12:42.124913022Z No resources found
2026-04-17T18:12:42.323122921Z ERROR: No running kube-apiserver pods found
2026-04-17T18:12:42.361713732Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-17T18:12:42.372796907Z error: the server doesn't have a resource type "performanceprofile"
2026-04-17T18:12:42.414112379Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-17T18:12:42.414112379Z See 'oc rsync -h' for help and examples
2026-04-17T18:12:42.415908798Z No resources found in openshift-etcd namespace.
2026-04-17T18:12:42.581488749Z INFO: Worker host service log collection to complete.
2026-04-17T18:12:42.593022203Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-17T18:12:42.791974072Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-17T18:12:42.828947651Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.181866841Z tar: Removing leading `/' from member names
2026-04-17T18:12:43.191252712Z INFO: HAProxy config collection complete.
2026-04-17T18:12:43.255185824Z No resources found
2026-04-17T18:12:43.274673958Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-17T18:12:43.329072343Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T18:12:43.375828236Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.387074263Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.396075183Z Gathering data for ns/openshift-monitoring...
2026-04-17T18:12:43.402149934Z INFO: 'previous-logs' folder not found on ip-10-0-130-19.ec2.internal, skipping...
2026-04-17T18:12:43.418693224Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.458948117Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.478450743Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.570276344Z error: the server doesn't have a resource type "machineconfigs"
2026-04-17T18:12:43.676115078Z INFO: 'previous-logs' folder not found on ip-10-0-136-202.ec2.internal, skipping...
2026-04-17T18:12:43.686828547Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T18:12:43.735947654Z INFO: Getting rules from prometheus-k8s-0
2026-04-17T18:12:43.746971312Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.809128025Z Wrote inspect data to must-gather.
2026-04-17T18:12:43.940798650Z INFO: 'previous-logs' folder not found on ip-10-0-140-33.ec2.internal, skipping...
2026-04-17T18:12:43.940841715Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-17T18:12:43.940873244Z INFO: Machine Config Daemon termination log collection complete.
2026-04-17T18:12:43.999106948Z Wrote inspect data to must-gather.
2026-04-17T18:12:44.038562895Z Wrote inspect data to must-gather.
2026-04-17T18:12:44.132359624Z INFO: Getting status/config from prometheus-k8s-0
2026-04-17T18:12:44.209811285Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-17T18:12:44.245495640Z Wrote inspect data to must-gather.
2026-04-17T18:12:44.430765359Z Wrote inspect data to must-gather.
2026-04-17T18:12:44.518719565Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-17T18:12:44.550136890Z INFO: INTERCONNECT MODE
2026-04-17T18:12:44.550236967Z INFO: Gathering ovn-kubernetes DBs
2026-04-17T18:12:44.675647337Z Wrote inspect data to must-gather.
2026-04-17T18:12:44.708883160Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T18:12:44.733992721Z INFO: Gathering OVN_Northbound from ovnkube-node-42tv5...
2026-04-17T18:12:44.734205174Z INFO: Gathering OVN_Northbound from ovnkube-node-p2pbl...
2026-04-17T18:12:44.734379801Z INFO: Gathering OVN_Northbound from ovnkube-node-rjptt...
2026-04-17T18:12:44.734648411Z INFO: Gathering OVN_Southbound from ovnkube-node-42tv5...
2026-04-17T18:12:44.734903816Z INFO: Gathering OVN_Southbound from ovnkube-node-p2pbl...
2026-04-17T18:12:44.735149244Z INFO: Gathering OVN_Southbound from ovnkube-node-rjptt...
2026-04-17T18:12:44.797830112Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-17T18:12:45.010854197Z tar: Removing leading `/' from member names
2026-04-17T18:12:45.035647650Z Wrote inspect data to must-gather.
2026-04-17T18:12:45.057054715Z tar: Removing leading `/' from member names
2026-04-17T18:12:45.063446806Z tar: Removing leading `/' from member names
2026-04-17T18:12:45.082038761Z tar: Removing leading `/' from member names
2026-04-17T18:12:45.082427097Z tar: Removing leading `/' from member names
2026-04-17T18:12:45.086769956Z tar: Removing leading `/' from member names
2026-04-17T18:12:45.168451408Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-17T18:12:45.302439650Z Wrote inspect data to must-gather.
2026-04-17T18:12:45.429252271Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-17T18:12:45.493025279Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T18:12:45.552932494Z INFO: Waiting for network log collection to complete ...
2026-04-17T18:12:45.553084973Z INFO: Waiting for ovnk database copies to complete ...
2026-04-17T18:12:45.553347321Z INFO: Copying ovnk databases complete.
2026-04-17T18:12:45.555389219Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-17T18:12:45.557366911Z ovnk_database_store/
2026-04-17T18:12:45.557366911Z ovnk_database_store/ovnkube-node-p2pbl_nbdb
2026-04-17T18:12:45.569634066Z ovnk_database_store/ovnkube-node-42tv5_sbdb
2026-04-17T18:12:45.590832172Z ovnk_database_store/ovnkube-node-p2pbl_sbdb
2026-04-17T18:12:45.612475424Z ovnk_database_store/ovnkube-node-rjptt_sbdb
2026-04-17T18:12:45.615765463Z Wrote inspect data to must-gather.
2026-04-17T18:12:45.636661099Z ovnk_database_store/ovnkube-node-42tv5_nbdb
2026-04-17T18:12:45.646986816Z ovnk_database_store/ovnkube-node-rjptt_nbdb
2026-04-17T18:12:45.662650524Z INFO: Network log collection complete.
2026-04-17T18:12:45.674143112Z Gathering data for ns/openshift-network-console...
2026-04-17T18:12:45.825427240Z INFO: Getting status from alertmanager-main-0
2026-04-17T18:12:45.913079081Z W0417 18:12:45.913022    1383 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-17T18:12:45.928212909Z Wrote inspect data to must-gather.
2026-04-17T18:12:46.093050629Z Gathering data for ns/openshift-console-operator...
2026-04-17T18:12:46.225999164Z Wrote inspect data to must-gather.
2026-04-17T18:12:46.446492017Z Gathering data for ns/openshift-console...
2026-04-17T18:12:46.478670321Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-17T18:12:46.480777254Z INFO: Fallback to identify the container image from release info
2026-04-17T18:12:46.854439011Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-17T18:12:47.008725579Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-17T18:12:47.139478653Z daemonset.apps/perf-node-gather-daemonset created
2026-04-17T18:12:47.222233778Z Gathering data for ns/openshift-dns-operator...
2026-04-17T18:12:47.285550616Z Waiting for performance profile collector pods to become ready: 1
2026-04-17T18:12:47.567399655Z Gathering data for ns/openshift-dns...
2026-04-17T18:12:48.191534233Z Gathering data for ns/openshift-image-registry...
2026-04-17T18:12:48.420421680Z Waiting for performance profile collector pods to become ready: 2
2026-04-17T18:12:48.660832564Z Gathering data for ns/openshift-ingress-operator...
2026-04-17T18:12:48.693364497Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T18:12:48.961357862Z Gathering data for ns/openshift-ingress...
2026-04-17T18:12:49.261979463Z Gathering data for ns/openshift-ingress-canary...
2026-04-17T18:12:49.557698991Z Waiting for performance profile collector pods to become ready: 3
2026-04-17T18:12:49.636213624Z Gathering data for ns/openshift-insights...
2026-04-17T18:12:50.680519958Z Waiting for performance profile collector pods to become ready: 4
2026-04-17T18:12:51.677546294Z Gathering data for ns/kserve...
2026-04-17T18:12:51.802334539Z Waiting for performance profile collector pods to become ready: 5
2026-04-17T18:12:52.925200285Z Waiting for performance profile collector pods to become ready: 6
2026-04-17T18:12:53.241247221Z Gathering data for ns/openshift-config...
2026-04-17T18:12:53.483871672Z Gathering data for ns/openshift-config-managed...
2026-04-17T18:12:53.697981333Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T18:12:53.902516529Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-17T18:12:54.055246067Z Waiting for performance profile collector pods to become ready: 7
2026-04-17T18:12:54.155135641Z Gathering data for ns/openshift-kube-apiserver...
2026-04-17T18:12:54.413712565Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-17T18:12:54.702105063Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-17T18:12:54.957840881Z Gathering data for ns/openshift-kube-scheduler...
2026-04-17T18:12:55.184246373Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-17T18:12:55.259308896Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-17T18:12:55.514872889Z Collecting performance related data for node ip-10-0-136-202.ec2.internal
2026-04-17T18:12:55.524657526Z Collecting performance related data for node ip-10-0-130-19.ec2.internal
2026-04-17T18:12:55.531006642Z Collecting performance related data for node ip-10-0-140-33.ec2.internal
2026-04-17T18:12:55.599480389Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-17T18:12:55.964874353Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-17T18:12:56.307859911Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-17T18:12:56.942261525Z Gathering data for ns/openshift-multus...
2026-04-17T18:12:58.326958318Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-17T18:12:58.702528798Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-17T18:12:59.984454491Z Gathering data for ns/openshift-host-network...
2026-04-17T18:13:00.272675791Z Gathering data for ns/openshift-network-diagnostics...
2026-04-17T18:13:00.778757247Z Gathering data for ns/openshift-network-node-identity...
2026-04-17T18:13:01.181486784Z Gathering data for ns/openshift-network-operator...
2026-04-17T18:13:01.526213051Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-17T18:13:01.786480813Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-17T18:13:02.199262765Z Gathering data for ns/openshift-apiserver-operator...
2026-04-17T18:13:02.516211352Z Gathering data for ns/openshift-apiserver...
2026-04-17T18:13:02.901094596Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-17T18:13:03.147476786Z Gathering data for ns/openshift-controller-manager...
2026-04-17T18:13:03.397450749Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-17T18:13:03.708790342Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-04-17T18:13:04.037405952Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-17T18:13:04.107210238Z Collecting kubelet logs for node ip-10-0-136-202.ec2.internal
2026-04-17T18:13:04.108901491Z Collecting kubelet logs for node ip-10-0-130-19.ec2.internal
2026-04-17T18:13:04.110600613Z Collecting kubelet logs for node ip-10-0-140-33.ec2.internal
2026-04-17T18:13:04.351685573Z Gathering data for ns/openshift-service-ca-operator...
2026-04-17T18:13:04.519710421Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-17T18:13:04.524918485Z INFO: Node performance data collection complete.
2026-04-17T18:13:04.661664650Z Gathering data for ns/openshift-service-ca...
2026-04-17T18:13:05.005761732Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-17T18:13:05.836369739Z Wrote inspect data to must-gather.
2026-04-17T18:13:05.836417538Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T18:13:05.836417538Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-17T18:13:05.968643476Z error: the server doesn't have a resource type "clusters"
2026-04-17T18:13:06.117453919Z Caches written to disk