2026-05-20T16:17:31.412544627Z [disk usage checker] Started
2026-05-20T16:17:31.416153912Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:17:33.710904539Z Gathering data for ns/openshift-cluster-version...
2026-05-20T16:17:33.861851392Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-20T16:17:33.952721302Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T16:17:34.051965709Z Gathering data for ns/default...
2026-05-20T16:17:34.378707673Z Gathering data for ns/openshift...
2026-05-20T16:17:34.840217222Z Gathering data for ns/kube-system...
2026-05-20T16:17:35.536993229Z Gathering data for ns/openshift-etcd...
2026-05-20T16:17:35.912748362Z Wrote inspect data to must-gather.
2026-05-20T16:17:35.912784928Z error: inspection completed with the errors occurred while gathering data:
2026-05-20T16:17:35.912784928Z     namespaces "assisted-installer" not found
2026-05-20T16:17:36.244737260Z Waiting on subprocesses to finish execution.
2026-05-20T16:17:36.250046988Z INFO: Collecting host service logs for crio
2026-05-20T16:17:36.250614728Z INFO: Collecting host service logs for kubelet
2026-05-20T16:17:36.250936215Z INFO: Collecting host service logs for rpm-ostreed
2026-05-20T16:17:36.251264201Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-20T16:17:36.251611388Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-20T16:17:36.251958373Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-20T16:17:36.252293437Z INFO: Collecting host service logs for NetworkManager
2026-05-20T16:17:36.252648815Z INFO: Collecting host service logs for openvswitch
2026-05-20T16:17:36.252981892Z INFO: Collecting host service logs for ovs-configuration
2026-05-20T16:17:36.253404033Z INFO: Collecting host service logs for ovsdb-server
2026-05-20T16:17:36.253839034Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-20T16:17:36.253877126Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-20T16:17:36.254277335Z INFO: Waiting for worker host service log collection to complete ...
2026-05-20T16:17:36.254365784Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-20T16:17:36.262956029Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-20T16:17:36.263712417Z INFO: Gathering HAProxy config files
2026-05-20T16:17:36.335677696Z INFO: Waiting for node performance related collection to complete ...
2026-05-20T16:17:36.537735708Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:17:36.658948981Z INFO: "metallb-operator" not detected. Skipping.
2026-05-20T16:17:36.691669526Z No resources found
2026-05-20T16:17:36.819866063Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-20T16:17:36.888029504Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-20T16:17:37.157068883Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-20T16:17:37.356126126Z error: the server doesn't have a resource type "performanceprofile"
2026-05-20T16:17:37.529800910Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-d5829
2026-05-20T16:17:37.529848452Z insights-runtime-extractor-6xcj5
2026-05-20T16:17:37.529867594Z insights-runtime-extractor-75mp8
2026-05-20T16:17:37.529887131Z insights-runtime-extractor-p574b
2026-05-20T16:17:37.688642284Z Wrote inspect data to must-gather.
2026-05-20T16:17:37.713653701Z No resources found in openshift-etcd namespace.
2026-05-20T16:17:37.846552725Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-20T16:17:37.899898785Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-20T16:17:37.899988638Z INFO: on-disk MachineConfig config collection complete.
2026-05-20T16:17:38.021676814Z ERROR: No running kube-apiserver pods found
2026-05-20T16:17:38.153410288Z INFO: Worker host service log collection to complete.
2026-05-20T16:17:38.210490326Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-20T16:17:38.504786636Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-20T16:17:38.643619614Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T16:17:38.707374277Z Wrote inspect data to must-gather.
2026-05-20T16:17:38.787650157Z Wrote inspect data to must-gather.
2026-05-20T16:17:38.941077477Z Wrote inspect data to must-gather.
2026-05-20T16:17:39.046553851Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-20T16:17:39.046553851Z See 'oc rsync -h' for help and examples
2026-05-20T16:17:39.119212413Z Wrote inspect data to must-gather.
2026-05-20T16:17:39.175055573Z No resources found
2026-05-20T16:17:39.210969518Z INFO: 'previous-logs' folder not found on ip-10-0-130-227.ec2.internal, skipping...
2026-05-20T16:17:39.230118472Z tar: Removing leading `/' from member names
2026-05-20T16:17:39.237513044Z INFO: HAProxy config collection complete.
2026-05-20T16:17:39.258344494Z Wrote inspect data to must-gather.
2026-05-20T16:17:39.282520064Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-20T16:17:39.334801070Z error: the server doesn't have a resource type "machineconfigs"
2026-05-20T16:17:39.545159046Z Gathering data for ns/kuadrant-system...
2026-05-20T16:17:39.572611695Z INFO: Getting rules from prometheus-k8s-0
2026-05-20T16:17:39.649671226Z INFO: 'previous-logs' folder not found on ip-10-0-138-16.ec2.internal, skipping...
2026-05-20T16:17:39.683046121Z Wrote inspect data to must-gather.
2026-05-20T16:17:39.731542233Z Wrote inspect data to must-gather.
2026-05-20T16:17:40.109199362Z INFO: 'previous-logs' folder not found on ip-10-0-141-222.ec2.internal, skipping...
2026-05-20T16:17:40.109697094Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-20T16:17:40.109697094Z INFO: Machine Config Daemon termination log collection complete.
2026-05-20T16:17:40.137308340Z Wrote inspect data to must-gather.
2026-05-20T16:17:40.141188486Z Wrote inspect data to must-gather.
2026-05-20T16:17:40.146889805Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-20T16:17:40.200270658Z INFO: Getting status/config from prometheus-k8s-0
2026-05-20T16:17:40.384565667Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-20T16:17:40.484418690Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-20T16:17:40.574935935Z Wrote inspect data to must-gather.
2026-05-20T16:17:40.717662899Z Gathering data for ns/openshift-monitoring...
2026-05-20T16:17:40.770251105Z Wrote inspect data to must-gather.
2026-05-20T16:17:40.847643706Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-20T16:17:41.002973955Z INFO: INTERCONNECT MODE
2026-05-20T16:17:41.003383904Z INFO: Gathering ovn-kubernetes DBs
2026-05-20T16:17:41.111788709Z Wrote inspect data to must-gather.
2026-05-20T16:17:41.149143829Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-20T16:17:41.223705656Z INFO: Gathering OVN_Northbound from ovnkube-node-dbwqx...
2026-05-20T16:17:41.223974013Z INFO: Gathering OVN_Northbound from ovnkube-node-gj2r6...
2026-05-20T16:17:41.224306729Z INFO: Gathering OVN_Northbound from ovnkube-node-tjrxd...
2026-05-20T16:17:41.224785927Z INFO: Gathering OVN_Southbound from ovnkube-node-dbwqx...
2026-05-20T16:17:41.225097095Z INFO: Gathering OVN_Southbound from ovnkube-node-gj2r6...
2026-05-20T16:17:41.225710486Z INFO: Gathering OVN_Southbound from ovnkube-node-tjrxd...
2026-05-20T16:17:41.565560444Z tar: Removing leading `/' from member names
2026-05-20T16:17:41.567858790Z tar: Removing leading `/' from member names
2026-05-20T16:17:41.613067683Z tar: Removing leading `/' from member names
2026-05-20T16:17:41.634562259Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-20T16:17:41.645066364Z tar: Removing leading `/' from member names
2026-05-20T16:17:41.677004348Z Wrote inspect data to must-gather.
2026-05-20T16:17:41.688703217Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:17:41.768274628Z tar: Removing leading `/' from member names
2026-05-20T16:17:41.769792104Z tar: Removing leading `/' from member names
2026-05-20T16:17:41.926876400Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-20T16:17:42.065038218Z Wrote inspect data to must-gather.
2026-05-20T16:17:42.396643782Z INFO: Waiting for network log collection to complete ...
2026-05-20T16:17:42.398029093Z INFO: Waiting for ovnk database copies to complete ...
2026-05-20T16:17:42.398029093Z INFO: Copying ovnk databases complete.
2026-05-20T16:17:42.399421265Z 5.0M	must-gather/network_logs/ovnk_database_store
2026-05-20T16:17:42.405160660Z ovnk_database_store/
2026-05-20T16:17:42.405160660Z ovnk_database_store/ovnkube-node-tjrxd_sbdb
2026-05-20T16:17:42.412522502Z Wrote inspect data to must-gather.
2026-05-20T16:17:42.436117747Z INFO: Getting status from alertmanager-main-0
2026-05-20T16:17:42.437371165Z ovnk_database_store/ovnkube-node-tjrxd_nbdb
2026-05-20T16:17:42.447833271Z ovnk_database_store/ovnkube-node-gj2r6_sbdb
2026-05-20T16:17:42.478567313Z ovnk_database_store/ovnkube-node-gj2r6_nbdb
2026-05-20T16:17:42.491317353Z ovnk_database_store/ovnkube-node-dbwqx_sbdb
2026-05-20T16:17:42.525651414Z ovnk_database_store/ovnkube-node-dbwqx_nbdb
2026-05-20T16:17:42.543644961Z INFO: Network log collection complete.
2026-05-20T16:17:42.741320107Z Wrote inspect data to must-gather.
2026-05-20T16:17:43.071615228Z Gathering data for ns/openshift-network-console...
2026-05-20T16:17:43.109296147Z Wrote inspect data to must-gather.
2026-05-20T16:17:43.442422774Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-20T16:17:43.447485886Z INFO: Fallback to identify the container image from release info
2026-05-20T16:17:43.611831722Z Gathering data for ns/openshift-console-operator...
2026-05-20T16:17:43.984747306Z Wrote inspect data to must-gather.
2026-05-20T16:17:44.020205642Z Gathering data for ns/openshift-console...
2026-05-20T16:17:44.102806417Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-20T16:17:44.245806798Z daemonset.apps/perf-node-gather-daemonset created
2026-05-20T16:17:44.400203429Z Waiting for performance profile collector pods to become ready: 1
2026-05-20T16:17:44.507701893Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-20T16:17:44.902171566Z Gathering data for ns/openshift-dns-operator...
2026-05-20T16:17:45.205039626Z Gathering data for ns/openshift-dns...
2026-05-20T16:17:45.546778882Z Waiting for performance profile collector pods to become ready: 2
2026-05-20T16:17:45.810990754Z Gathering data for ns/openshift-image-registry...
2026-05-20T16:17:46.285331364Z Gathering data for ns/openshift-ingress-operator...
2026-05-20T16:17:46.667555763Z Gathering data for ns/openshift-ingress...
2026-05-20T16:17:46.697632342Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:17:46.704490823Z Waiting for performance profile collector pods to become ready: 3
2026-05-20T16:17:47.240162612Z Gathering data for ns/openshift-ingress-canary...
2026-05-20T16:17:47.711412145Z Gathering data for ns/openshift-insights...
2026-05-20T16:17:47.840541290Z Waiting for performance profile collector pods to become ready: 4
2026-05-20T16:17:48.967393130Z Waiting for performance profile collector pods to become ready: 5
2026-05-20T16:17:50.095170006Z Waiting for performance profile collector pods to become ready: 6
2026-05-20T16:17:50.392249715Z Gathering data for ns/openshift-lws-operator...
2026-05-20T16:17:50.924966768Z Gathering data for ns/kserve...
2026-05-20T16:17:51.247534921Z Waiting for performance profile collector pods to become ready: 7
2026-05-20T16:17:51.702825981Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:17:52.381175860Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-20T16:17:52.675190606Z Gathering data for ns/openshift-config...
2026-05-20T16:17:52.676859454Z Collecting performance related data for node ip-10-0-130-227.ec2.internal
2026-05-20T16:17:52.685293305Z Collecting performance related data for node ip-10-0-138-16.ec2.internal
2026-05-20T16:17:52.699729751Z Collecting performance related data for node ip-10-0-141-222.ec2.internal
2026-05-20T16:17:53.024332447Z Gathering data for ns/openshift-config-managed...
2026-05-20T16:17:53.603528135Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-20T16:17:53.990024674Z Gathering data for ns/openshift-kube-apiserver...
2026-05-20T16:17:54.347642309Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-20T16:17:54.714733554Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-20T16:17:55.038312100Z Gathering data for ns/openshift-kube-scheduler...
2026-05-20T16:17:55.352323716Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-20T16:17:55.747627055Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-20T16:17:56.123249002Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-20T16:17:56.463067638Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-20T16:17:56.712357215Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:17:57.068201794Z Gathering data for ns/openshift-multus...
2026-05-20T16:17:58.485389128Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-20T16:18:00.170212253Z Gathering data for ns/openshift-host-network...
2026-05-20T16:18:00.526486232Z Gathering data for ns/openshift-network-diagnostics...
2026-05-20T16:18:01.091745584Z Gathering data for ns/openshift-network-node-identity...
2026-05-20T16:18:01.541455201Z Gathering data for ns/openshift-network-operator...
2026-05-20T16:18:01.717518052Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:18:01.939458709Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-20T16:18:02.079768969Z Collecting kubelet logs for node ip-10-0-130-227.ec2.internal
2026-05-20T16:18:02.085619009Z Collecting kubelet logs for node ip-10-0-138-16.ec2.internal
2026-05-20T16:18:02.087759639Z Collecting kubelet logs for node ip-10-0-141-222.ec2.internal
2026-05-20T16:18:02.294998977Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-20T16:18:02.535343209Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-20T16:18:02.541048214Z INFO: Node performance data collection complete.
2026-05-20T16:18:02.815509538Z Gathering data for ns/openshift-apiserver-operator...
2026-05-20T16:18:03.133053679Z Gathering data for ns/openshift-apiserver...
2026-05-20T16:18:03.554195641Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-20T16:18:03.851021655Z Gathering data for ns/openshift-controller-manager...
2026-05-20T16:18:04.150953280Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-20T16:18:04.813570519Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-20T16:18:05.110827274Z Gathering data for ns/openshift-service-ca-operator...
2026-05-20T16:18:05.428550814Z Gathering data for ns/openshift-service-ca...
2026-05-20T16:18:05.812030499Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-20T16:18:06.722466131Z [disk usage checker] Volume usage percentage: current = 23 ; allowed = 70
2026-05-20T16:18:06.742342209Z Wrote inspect data to must-gather.
2026-05-20T16:18:06.742389867Z error: inspection completed with the errors occurred while gathering data:
2026-05-20T16:18:06.742389867Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-20T16:18:06.890999891Z error: the server doesn't have a resource type "clusters"
2026-05-20T16:18:07.039001458Z Caches written to disk