2026-04-16T17:10:57.304797920Z [disk usage checker] Started
2026-04-16T17:10:57.308137515Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T17:10:57.526736282Z Gathering data for ns/openshift-cluster-version...
2026-04-16T17:10:57.676974929Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T17:10:57.762007893Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T17:10:57.847051493Z Gathering data for ns/default...
2026-04-16T17:10:58.134644939Z Gathering data for ns/openshift...
2026-04-16T17:10:58.528476835Z Gathering data for ns/kube-system...
2026-04-16T17:10:59.169234736Z Gathering data for ns/openshift-etcd...
2026-04-16T17:10:59.472176585Z Wrote inspect data to must-gather.
2026-04-16T17:10:59.472217855Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T17:10:59.472217855Z     namespaces "assisted-installer" not found
2026-04-16T17:10:59.769402781Z Waiting on subprocesses to finish execution.
2026-04-16T17:10:59.776921159Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-16T17:10:59.782061644Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-16T17:10:59.782298523Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-16T17:10:59.786622255Z INFO: Gathering HAProxy config files
2026-04-16T17:10:59.790692328Z INFO: Collecting host service logs for crio
2026-04-16T17:10:59.791289217Z INFO: Collecting host service logs for kubelet
2026-04-16T17:10:59.791648749Z INFO: Collecting host service logs for rpm-ostreed
2026-04-16T17:10:59.792007430Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-16T17:10:59.792449409Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-16T17:10:59.792837696Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-16T17:10:59.793174671Z INFO: Collecting host service logs for NetworkManager
2026-04-16T17:10:59.793510468Z INFO: Collecting host service logs for openvswitch
2026-04-16T17:10:59.793889096Z INFO: Collecting host service logs for ovs-configuration
2026-04-16T17:10:59.794227714Z INFO: Collecting host service logs for ovsdb-server
2026-04-16T17:10:59.794603622Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-16T17:10:59.795095605Z INFO: Waiting for worker host service log collection to complete ...
2026-04-16T17:10:59.853390179Z INFO: Waiting for node performance related collection to complete ...
2026-04-16T17:11:00.366281132Z INFO: Collecting Insights Archives from insights-operator-5785d4fcdd-jjnfw
2026-04-16T17:11:00.366281132Z insights-runtime-extractor-ctpwk
2026-04-16T17:11:00.366281132Z insights-runtime-extractor-sqgtm
2026-04-16T17:11:00.366281132Z insights-runtime-extractor-tjmzf
2026-04-16T17:11:00.563361288Z No resources found in openshift-etcd namespace.
2026-04-16T17:11:00.735129736Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-16T17:11:00.903051288Z INFO: "metallb-operator" not detected. Skipping.
2026-04-16T17:11:00.933015300Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-16T17:11:00.997895721Z Wrote inspect data to must-gather.
2026-04-16T17:11:01.030096445Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-16T17:11:01.030130389Z INFO: on-disk MachineConfig config collection complete.
2026-04-16T17:11:01.062361175Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-16T17:11:01.066630652Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-16T17:11:01.085094815Z INFO: Worker host service log collection to complete.
2026-04-16T17:11:01.160578256Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-16T17:11:01.173576447Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-16T17:11:01.173576447Z See 'oc rsync -h' for help and examples
2026-04-16T17:11:01.181340158Z No resources found
2026-04-16T17:11:01.281338977Z ERROR: No running kube-apiserver pods found
2026-04-16T17:11:01.316149896Z error: the server doesn't have a resource type "performanceprofile"
2026-04-16T17:11:01.517674033Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-16T17:11:01.789039778Z No resources found
2026-04-16T17:11:01.894824313Z Wrote inspect data to must-gather.
2026-04-16T17:11:01.926703016Z Wrote inspect data to must-gather.
2026-04-16T17:11:01.943908888Z tar: Removing leading `/' from member names
2026-04-16T17:11:01.959696601Z INFO: HAProxy config collection complete.
2026-04-16T17:11:01.994686336Z Gathering data for ns/openshift-monitoring...
2026-04-16T17:11:02.004021027Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T17:11:02.024033970Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-16T17:11:02.054848850Z INFO: 'previous-logs' folder not found on ip-10-0-128-130.ec2.internal, skipping...
2026-04-16T17:11:02.071877840Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.120213112Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.166734369Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.168398158Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.318856181Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T17:11:02.357660775Z error: the server doesn't have a resource type "machineconfigs"
2026-04-16T17:11:02.359978768Z INFO: 'previous-logs' folder not found on ip-10-0-130-1.ec2.internal, skipping...
2026-04-16T17:11:02.387975800Z INFO: Getting rules from prometheus-k8s-0
2026-04-16T17:11:02.439727725Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.599322078Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.626048109Z INFO: 'previous-logs' folder not found on ip-10-0-131-63.ec2.internal, skipping...
2026-04-16T17:11:02.626078097Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-16T17:11:02.626105722Z INFO: Machine Config Daemon termination log collection complete.
2026-04-16T17:11:02.692219573Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.793607399Z INFO: Getting status/config from prometheus-k8s-0
2026-04-16T17:11:02.813475038Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.939938768Z Wrote inspect data to must-gather.
2026-04-16T17:11:02.981401560Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-16T17:11:03.207335572Z Wrote inspect data to must-gather.
2026-04-16T17:11:03.225644763Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-16T17:11:03.241305625Z INFO: INTERCONNECT MODE
2026-04-16T17:11:03.241339969Z INFO: Gathering ovn-kubernetes DBs
2026-04-16T17:11:03.438919089Z INFO: Gathering OVN_Northbound from ovnkube-node-5pfjs...
2026-04-16T17:11:03.439177046Z INFO: Gathering OVN_Northbound from ovnkube-node-6dbws...
2026-04-16T17:11:03.439403822Z INFO: Gathering OVN_Northbound from ovnkube-node-s2x9b...
2026-04-16T17:11:03.439707017Z INFO: Gathering OVN_Southbound from ovnkube-node-5pfjs...
2026-04-16T17:11:03.440022933Z INFO: Gathering OVN_Southbound from ovnkube-node-6dbws...
2026-04-16T17:11:03.440373393Z INFO: Gathering OVN_Southbound from ovnkube-node-s2x9b...
2026-04-16T17:11:03.457171038Z Wrote inspect data to must-gather.
2026-04-16T17:11:03.518776921Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-16T17:11:03.533415874Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-16T17:11:03.732119255Z tar: Removing leading `/' from member names
2026-04-16T17:11:03.741608669Z tar: Removing leading `/' from member names
2026-04-16T17:11:03.744928260Z tar: Removing leading `/' from member names
2026-04-16T17:11:03.760952529Z tar: Removing leading `/' from member names
2026-04-16T17:11:03.769960420Z tar: Removing leading `/' from member names
2026-04-16T17:11:03.780627600Z tar: Removing leading `/' from member names
2026-04-16T17:11:03.837881448Z Wrote inspect data to must-gather.
2026-04-16T17:11:03.867986811Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-16T17:11:04.109481274Z Wrote inspect data to must-gather.
2026-04-16T17:11:04.135794980Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-16T17:11:04.250977385Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-16T17:11:04.305022336Z INFO: Waiting for network log collection to complete ...
2026-04-16T17:11:04.305066004Z INFO: Waiting for ovnk database copies to complete ...
2026-04-16T17:11:04.305576887Z INFO: Copying ovnk databases complete.
2026-04-16T17:11:04.307714181Z 3.3M	must-gather/network_logs/ovnk_database_store
2026-04-16T17:11:04.311581305Z ovnk_database_store/
2026-04-16T17:11:04.311634245Z ovnk_database_store/ovnkube-node-5pfjs_sbdb
2026-04-16T17:11:04.333937712Z ovnk_database_store/ovnkube-node-6dbws_sbdb
2026-04-16T17:11:04.354435009Z ovnk_database_store/ovnkube-node-6dbws_nbdb
2026-04-16T17:11:04.364999591Z ovnk_database_store/ovnkube-node-s2x9b_sbdb
2026-04-16T17:11:04.383685613Z ovnk_database_store/ovnkube-node-5pfjs_nbdb
2026-04-16T17:11:04.393148766Z ovnk_database_store/ovnkube-node-s2x9b_nbdb
2026-04-16T17:11:04.404825372Z INFO: Network log collection complete.
2026-04-16T17:11:04.434230581Z Wrote inspect data to must-gather.
2026-04-16T17:11:04.438130841Z Gathering data for ns/openshift-network-console...
2026-04-16T17:11:04.560278729Z INFO: Getting status from alertmanager-main-0
2026-04-16T17:11:04.743508504Z Wrote inspect data to must-gather.
2026-04-16T17:11:04.861228729Z Gathering data for ns/openshift-console-operator...
2026-04-16T17:11:05.027844905Z Wrote inspect data to must-gather.
2026-04-16T17:11:05.224081670Z Gathering data for ns/openshift-console...
2026-04-16T17:11:05.309825091Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-16T17:11:05.312686496Z INFO: Fallback to identify the container image from release info
2026-04-16T17:11:05.639618674Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-16T17:11:05.955893631Z Gathering data for ns/openshift-dns-operator...
2026-04-16T17:11:06.251707401Z Gathering data for ns/openshift-dns...
2026-04-16T17:11:06.520748307Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:fc77fc5f1546189d6b57967ccc684e794bd846ea36c3e3c1e61d83a684f0f9b9
2026-04-16T17:11:06.640021983Z daemonset.apps/perf-node-gather-daemonset created
2026-04-16T17:11:06.789879849Z Waiting for performance profile collector pods to become ready: 1
2026-04-16T17:11:06.846651280Z Gathering data for ns/openshift-image-registry...
2026-04-16T17:11:07.296077586Z Gathering data for ns/openshift-ingress-operator...
2026-04-16T17:11:07.326463350Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T17:11:07.606885031Z Gathering data for ns/openshift-ingress...
2026-04-16T17:11:07.931290476Z Waiting for performance profile collector pods to become ready: 2
2026-04-16T17:11:07.931339589Z Gathering data for ns/openshift-ingress-canary...
2026-04-16T17:11:08.290042277Z Gathering data for ns/openshift-insights...
2026-04-16T17:11:09.048953407Z Waiting for performance profile collector pods to become ready: 3
2026-04-16T17:11:10.168636866Z Waiting for performance profile collector pods to become ready: 4
2026-04-16T17:11:10.330705709Z Gathering data for ns/kserve...
2026-04-16T17:11:11.288063771Z Waiting for performance profile collector pods to become ready: 5
2026-04-16T17:11:11.705782586Z Gathering data for ns/openshift-config...
2026-04-16T17:11:11.946707016Z Gathering data for ns/openshift-config-managed...
2026-04-16T17:11:12.332667135Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T17:11:12.394170367Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-16T17:11:12.422895957Z Waiting for performance profile collector pods to become ready: 6
2026-04-16T17:11:12.637117652Z Gathering data for ns/openshift-kube-apiserver...
2026-04-16T17:11:12.876381870Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-16T17:11:13.136014401Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-16T17:11:13.392382022Z Gathering data for ns/openshift-kube-scheduler...
2026-04-16T17:11:13.543550454Z Waiting for performance profile collector pods to become ready: 7
2026-04-16T17:11:13.669334610Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-16T17:11:13.956003637Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-16T17:11:14.288959524Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-16T17:11:14.583487978Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-16T17:11:14.680386222Z Waiting for performance profile collector pods to become ready: 8
2026-04-16T17:11:15.062218100Z Gathering data for ns/openshift-multus...
2026-04-16T17:11:15.807658383Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-16T17:11:16.085552530Z Collecting performance related data for node ip-10-0-131-63.ec2.internal
2026-04-16T17:11:16.093029174Z Collecting performance related data for node ip-10-0-128-130.ec2.internal
2026-04-16T17:11:16.101239276Z Collecting performance related data for node ip-10-0-130-1.ec2.internal
2026-04-16T17:11:16.505920430Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-16T17:11:17.346804204Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T17:11:18.053904225Z Gathering data for ns/openshift-host-network...
2026-04-16T17:11:18.333184187Z Gathering data for ns/openshift-network-diagnostics...
2026-04-16T17:11:18.927111651Z Gathering data for ns/openshift-network-node-identity...
2026-04-16T17:11:19.357762580Z Gathering data for ns/openshift-network-operator...
2026-04-16T17:11:19.708909313Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-16T17:11:19.955655038Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-16T17:11:20.419606549Z Gathering data for ns/openshift-apiserver-operator...
2026-04-16T17:11:20.713737333Z Gathering data for ns/openshift-apiserver...
2026-04-16T17:11:21.091873148Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-16T17:11:21.398884381Z Gathering data for ns/openshift-controller-manager...
2026-04-16T17:11:21.728226830Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-16T17:11:22.352122476Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-16T17:11:22.359888123Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-16T17:11:22.628562071Z Gathering data for ns/openshift-service-ca-operator...
2026-04-16T17:11:22.924250129Z Gathering data for ns/openshift-service-ca...
2026-04-16T17:11:23.290850784Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-16T17:11:24.112909912Z Collecting kubelet logs for node ip-10-0-131-63.ec2.internal
2026-04-16T17:11:24.114815166Z Collecting kubelet logs for node ip-10-0-128-130.ec2.internal
2026-04-16T17:11:24.117127599Z Collecting kubelet logs for node ip-10-0-130-1.ec2.internal
2026-04-16T17:11:24.198083756Z Wrote inspect data to must-gather.
2026-04-16T17:11:24.198127013Z error: inspection completed with the errors occurred while gathering data:
2026-04-16T17:11:24.198127013Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-16T17:11:24.446092009Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-16T17:11:24.450793299Z INFO: Node performance data collection complete.
2026-04-16T17:11:24.584122569Z error: the server doesn't have a resource type "clusters"
2026-04-16T17:11:24.742276149Z Caches written to disk