2026-04-20T21:47:01.364431615Z [disk usage checker] Started
2026-04-20T21:47:01.397344168Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-20T21:47:01.593910132Z Gathering data for ns/openshift-cluster-version...
2026-04-20T21:47:01.746164401Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T21:47:01.865589140Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T21:47:01.966093985Z Gathering data for ns/default...
2026-04-20T21:47:02.361424127Z Gathering data for ns/openshift...
2026-04-20T21:47:02.823157544Z Gathering data for ns/kube-system...
2026-04-20T21:47:03.684741043Z Gathering data for ns/openshift-etcd...
2026-04-20T21:47:03.751123964Z Waiting on subprocesses to finish execution.
2026-04-20T21:47:03.758688214Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-20T21:47:03.783785925Z INFO: Collecting host service logs for crio
2026-04-20T21:47:03.784202964Z INFO: Collecting host service logs for kubelet
2026-04-20T21:47:03.784567132Z INFO: Collecting host service logs for rpm-ostreed
2026-04-20T21:47:03.784899664Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-20T21:47:03.785312802Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-20T21:47:03.785628649Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-20T21:47:03.785810785Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-20T21:47:03.786323254Z INFO: Collecting host service logs for NetworkManager
2026-04-20T21:47:03.786666682Z INFO: Collecting host service logs for openvswitch
2026-04-20T21:47:03.786993405Z INFO: Collecting host service logs for ovs-configuration
2026-04-20T21:47:03.787355154Z INFO: Collecting host service logs for ovsdb-server
2026-04-20T21:47:03.787686473Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-20T21:47:03.788093605Z INFO: Waiting for worker host service log collection to complete ...
2026-04-20T21:47:03.797081556Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-20T21:47:03.813869051Z INFO: Gathering HAProxy config files
2026-04-20T21:47:03.829042232Z INFO: Waiting for node performance related collection to complete ...
2026-04-20T21:47:04.067551097Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-20T21:47:04.067629069Z INFO: on-disk MachineConfig config collection complete.
2026-04-20T21:47:04.432693625Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-20T21:47:04.457029011Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-20T21:47:04.542841656Z No resources found
2026-04-20T21:47:04.767687336Z error: the server doesn't have a resource type "performanceprofile"
2026-04-20T21:47:04.861365637Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-xxkhk
2026-04-20T21:47:04.861422608Z insights-runtime-extractor-9gggf
2026-04-20T21:47:04.861443646Z insights-runtime-extractor-kmpf4
2026-04-20T21:47:04.861463344Z insights-runtime-extractor-z4nxr
2026-04-20T21:47:04.958631026Z No resources found in openshift-etcd namespace.
2026-04-20T21:47:05.016499579Z Wrote inspect data to must-gather.
2026-04-20T21:47:05.114782324Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-20T21:47:05.210210890Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-20T21:47:05.430655174Z INFO: "metallb-operator" not detected. Skipping.
2026-04-20T21:47:05.459251826Z Wrote inspect data to must-gather.
2026-04-20T21:47:05.459275401Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T21:47:05.459275401Z     namespaces "assisted-installer" not found
2026-04-20T21:47:05.481461980Z INFO: Worker host service log collection to complete.
2026-04-20T21:47:05.928920548Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-20T21:47:06.058674467Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-20T21:47:06.058674467Z See 'oc rsync -h' for help and examples
2026-04-20T21:47:06.161297949Z Wrote inspect data to must-gather.
2026-04-20T21:47:06.192635109Z INFO: 'previous-logs' folder not found on ip-10-0-129-149.ec2.internal, skipping...
2026-04-20T21:47:06.200418146Z Wrote inspect data to must-gather.
2026-04-20T21:47:06.301416471Z Wrote inspect data to must-gather.
2026-04-20T21:47:06.330359570Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T21:47:06.363757406Z No resources found
2026-04-20T21:47:06.376585302Z Wrote inspect data to must-gather.
2026-04-20T21:47:06.404393794Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-20T21:47:06.423863523Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-20T21:47:06.443214067Z Wrote inspect data to must-gather.
2026-04-20T21:47:06.542973526Z tar: Removing leading `/' from member names
2026-04-20T21:47:06.546780618Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-20T21:47:06.546869334Z INFO: HAProxy config collection complete.
2026-04-20T21:47:06.557063394Z ERROR: No running kube-apiserver pods found
2026-04-20T21:47:06.687056169Z error: the server doesn't have a resource type "machineconfigs"
2026-04-20T21:47:06.873546164Z INFO: 'previous-logs' folder not found on ip-10-0-129-57.ec2.internal, skipping...
2026-04-20T21:47:06.875928098Z Wrote inspect data to must-gather.
2026-04-20T21:47:06.994296358Z Gathering data for ns/kuadrant-system...
2026-04-20T21:47:07.020270675Z Wrote inspect data to must-gather.
2026-04-20T21:47:07.123893301Z INFO: Getting rules from prometheus-k8s-0
2026-04-20T21:47:07.225707149Z Wrote inspect data to must-gather.
2026-04-20T21:47:07.239688429Z INFO: 'previous-logs' folder not found on ip-10-0-132-45.ec2.internal, skipping...
2026-04-20T21:47:07.239722580Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-20T21:47:07.239722580Z INFO: Machine Config Daemon termination log collection complete.
2026-04-20T21:47:07.331836487Z Wrote inspect data to must-gather.
2026-04-20T21:47:07.619300937Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-20T21:47:07.626648950Z Wrote inspect data to must-gather.
2026-04-20T21:47:07.652745702Z INFO: Getting status/config from prometheus-k8s-0
2026-04-20T21:47:07.901499419Z Wrote inspect data to must-gather.
2026-04-20T21:47:08.024534328Z INFO: INTERCONNECT MODE
2026-04-20T21:47:08.024578387Z INFO: Gathering ovn-kubernetes DBs
2026-04-20T21:47:08.143621706Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-20T21:47:08.190055630Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T21:47:08.248450023Z Wrote inspect data to must-gather.
2026-04-20T21:47:08.254706760Z INFO: Gathering OVN_Northbound from ovnkube-node-5qmcs...
2026-04-20T21:47:08.254943031Z INFO: Gathering OVN_Northbound from ovnkube-node-prfjl...
2026-04-20T21:47:08.255232127Z INFO: Gathering OVN_Northbound from ovnkube-node-q6skp...
2026-04-20T21:47:08.255716365Z INFO: Gathering OVN_Southbound from ovnkube-node-5qmcs...
2026-04-20T21:47:08.255997594Z INFO: Gathering OVN_Southbound from ovnkube-node-prfjl...
2026-04-20T21:47:08.256288458Z INFO: Gathering OVN_Southbound from ovnkube-node-q6skp...
2026-04-20T21:47:08.441908718Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-20T21:47:08.585936351Z tar: Removing leading `/' from member names
2026-04-20T21:47:08.657109167Z tar: Removing leading `/' from member names
2026-04-20T21:47:08.670783389Z tar: Removing leading `/' from member names
2026-04-20T21:47:08.709794392Z tar: Removing leading `/' from member names
2026-04-20T21:47:08.718381165Z tar: Removing leading `/' from member names
2026-04-20T21:47:08.734350445Z tar: Removing leading `/' from member names
2026-04-20T21:47:08.805754190Z Wrote inspect data to must-gather.
2026-04-20T21:47:08.864581416Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-20T21:47:08.952321390Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T21:47:09.174977144Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-20T21:47:09.214354904Z Gathering data for ns/openshift-monitoring...
2026-04-20T21:47:09.235302888Z Wrote inspect data to must-gather.
2026-04-20T21:47:09.453529920Z INFO: Waiting for network log collection to complete ...
2026-04-20T21:47:09.453558650Z INFO: Waiting for ovnk database copies to complete ...
2026-04-20T21:47:09.453913296Z INFO: Copying ovnk databases complete.
2026-04-20T21:47:09.455754067Z 4.2M	must-gather/network_logs/ovnk_database_store
2026-04-20T21:47:09.457913336Z ovnk_database_store/
2026-04-20T21:47:09.457913336Z ovnk_database_store/ovnkube-node-q6skp_nbdb
2026-04-20T21:47:09.471235005Z ovnk_database_store/ovnkube-node-5qmcs_sbdb
2026-04-20T21:47:09.493225883Z ovnk_database_store/ovnkube-node-prfjl_sbdb
2026-04-20T21:47:09.516528091Z ovnk_database_store/ovnkube-node-5qmcs_nbdb
2026-04-20T21:47:09.527490866Z ovnk_database_store/ovnkube-node-q6skp_sbdb
2026-04-20T21:47:09.549934469Z ovnk_database_store/ovnkube-node-prfjl_nbdb
2026-04-20T21:47:09.563834575Z INFO: Network log collection complete.
2026-04-20T21:47:09.604483822Z Wrote inspect data to must-gather.
2026-04-20T21:47:09.655002021Z INFO: Getting status from alertmanager-main-0
2026-04-20T21:47:09.895531463Z Wrote inspect data to must-gather.
2026-04-20T21:47:10.163525144Z Wrote inspect data to must-gather.
2026-04-20T21:47:10.481373457Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-20T21:47:10.484834227Z INFO: Fallback to identify the container image from release info
2026-04-20T21:47:11.141862494Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-20T21:47:11.217692554Z Gathering data for ns/openshift-network-console...
2026-04-20T21:47:11.260525287Z daemonset.apps/perf-node-gather-daemonset created
2026-04-20T21:47:11.418353977Z Waiting for performance profile collector pods to become ready: 1
2026-04-20T21:47:11.428898378Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-20T21:47:11.759875381Z Gathering data for ns/openshift-console-operator...
2026-04-20T21:47:12.196798900Z Gathering data for ns/openshift-console...
2026-04-20T21:47:12.556812996Z Waiting for performance profile collector pods to become ready: 2
2026-04-20T21:47:12.701859217Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-20T21:47:12.906029837Z Wrote inspect data to must-gather.
2026-04-20T21:47:13.063799678Z Gathering data for ns/openshift-dns-operator...
2026-04-20T21:47:13.384213846Z Gathering data for ns/openshift-dns...
2026-04-20T21:47:13.691249277Z Waiting for performance profile collector pods to become ready: 3
2026-04-20T21:47:13.977859914Z Gathering data for ns/openshift-image-registry...
2026-04-20T21:47:14.423565733Z Gathering data for ns/openshift-ingress-operator...
2026-04-20T21:47:14.804682162Z Gathering data for ns/openshift-ingress...
2026-04-20T21:47:14.816863702Z Waiting for performance profile collector pods to become ready: 4
2026-04-20T21:47:15.467039528Z Gathering data for ns/openshift-ingress-canary...
2026-04-20T21:47:15.895468437Z Gathering data for ns/openshift-insights...
2026-04-20T21:47:15.945121434Z Waiting for performance profile collector pods to become ready: 5
2026-04-20T21:47:16.433472125Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-20T21:47:17.050568820Z Waiting for performance profile collector pods to become ready: 6
2026-04-20T21:47:17.872140464Z Gathering data for ns/opendatahub...
2026-04-20T21:47:18.162656949Z Waiting for performance profile collector pods to become ready: 7
2026-04-20T21:47:19.230134020Z Gathering data for ns/openshift-lws-operator...
2026-04-20T21:47:19.278982586Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-20T21:47:19.532434592Z Collecting performance related data for node ip-10-0-129-57.ec2.internal
2026-04-20T21:47:19.540088645Z Collecting performance related data for node ip-10-0-129-149.ec2.internal
2026-04-20T21:47:19.547817111Z Collecting performance related data for node ip-10-0-132-45.ec2.internal
2026-04-20T21:47:20.633411287Z Gathering data for ns/openshift-config...
2026-04-20T21:47:21.044282600Z Gathering data for ns/openshift-config-managed...
2026-04-20T21:47:21.438425111Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-20T21:47:21.499871480Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-20T21:47:21.804707859Z Gathering data for ns/openshift-kube-apiserver...
2026-04-20T21:47:22.146861509Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-20T21:47:22.516395005Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-20T21:47:22.854997987Z Gathering data for ns/openshift-kube-scheduler...
2026-04-20T21:47:23.176700957Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-20T21:47:23.578114864Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-20T21:47:23.936640803Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-20T21:47:24.284751679Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-20T21:47:24.856788494Z Gathering data for ns/openshift-multus...
2026-04-20T21:47:26.211341875Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-20T21:47:26.444412069Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-20T21:47:27.614262855Z Gathering data for ns/openshift-host-network...
2026-04-20T21:47:27.862667772Z Collecting kubelet logs for node ip-10-0-129-57.ec2.internal
2026-04-20T21:47:27.865766282Z Collecting kubelet logs for node ip-10-0-129-149.ec2.internal
2026-04-20T21:47:27.868530030Z Collecting kubelet logs for node ip-10-0-132-45.ec2.internal
2026-04-20T21:47:27.962555666Z Gathering data for ns/openshift-network-diagnostics...
2026-04-20T21:47:28.240319784Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-20T21:47:28.244647403Z INFO: Node performance data collection complete.
2026-04-20T21:47:28.575995293Z Gathering data for ns/openshift-network-node-identity...
2026-04-20T21:47:28.987437947Z Gathering data for ns/openshift-network-operator...
2026-04-20T21:47:29.367641075Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-20T21:47:29.677953278Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-20T21:47:30.137513323Z Gathering data for ns/openshift-apiserver-operator...
2026-04-20T21:47:30.499340241Z Gathering data for ns/openshift-apiserver...
2026-04-20T21:47:30.931388871Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-20T21:47:31.227751689Z Gathering data for ns/openshift-controller-manager...
2026-04-20T21:47:31.448932774Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-20T21:47:31.533056378Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-20T21:47:32.151971116Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-20T21:47:32.443054815Z Gathering data for ns/openshift-service-ca-operator...
2026-04-20T21:47:32.757913653Z Gathering data for ns/openshift-service-ca...
2026-04-20T21:47:33.131879941Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-20T21:47:34.108290020Z Wrote inspect data to must-gather.
2026-04-20T21:47:34.108332765Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T21:47:34.108332765Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-20T21:47:34.235704421Z error: the server doesn't have a resource type "clusters"
2026-04-20T21:47:34.400845022Z Caches written to disk