2026-04-24T22:16:37.322122743Z [disk usage checker] Started
2026-04-24T22:16:37.353388117Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-24T22:16:37.533720508Z Gathering data for ns/openshift-cluster-version...
2026-04-24T22:16:37.671002160Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:16:37.764500104Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:16:37.845748050Z Gathering data for ns/default...
2026-04-24T22:16:38.141532206Z Gathering data for ns/openshift...
2026-04-24T22:16:38.538814521Z Gathering data for ns/kube-system...
2026-04-24T22:16:39.254319041Z Gathering data for ns/openshift-etcd...
2026-04-24T22:16:39.585196506Z Wrote inspect data to must-gather.
2026-04-24T22:16:39.585228918Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:16:39.585228918Z     namespaces "assisted-installer" not found
2026-04-24T22:16:39.745005483Z Waiting on subprocesses to finish execution.
2026-04-24T22:16:39.753750698Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T22:16:39.756759407Z INFO: Collecting host service logs for crio
2026-04-24T22:16:39.757134403Z INFO: Collecting host service logs for kubelet
2026-04-24T22:16:39.757482850Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T22:16:39.757660091Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T22:16:39.757863256Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T22:16:39.758216503Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T22:16:39.758537778Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T22:16:39.758867299Z INFO: Collecting host service logs for NetworkManager
2026-04-24T22:16:39.759265897Z INFO: Collecting host service logs for openvswitch
2026-04-24T22:16:39.759617277Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T22:16:39.759977093Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T22:16:39.760296295Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T22:16:39.760344547Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T22:16:39.760730360Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T22:16:39.763315503Z INFO: Gathering HAProxy config files
2026-04-24T22:16:39.805379716Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T22:16:40.462647499Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T22:16:40.466994949Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T22:16:40.497482573Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T22:16:40.578952137Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T22:16:40.579043371Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T22:16:40.637446227Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-lm6jb
2026-04-24T22:16:40.637446227Z insights-runtime-extractor-mvrpd
2026-04-24T22:16:40.637446227Z insights-runtime-extractor-p4ctn
2026-04-24T22:16:40.637446227Z insights-runtime-extractor-xwrsb
2026-04-24T22:16:40.696963330Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T22:16:40.755987403Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T22:16:40.898637665Z No resources found
2026-04-24T22:16:40.940481124Z Wrote inspect data to must-gather.
2026-04-24T22:16:41.010026101Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T22:16:41.251752851Z No resources found in openshift-etcd namespace.
2026-04-24T22:16:41.287406610Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T22:16:41.287406610Z See 'oc rsync -h' for help and examples
2026-04-24T22:16:41.392809151Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T22:16:41.419851253Z INFO: Worker host service log collection to complete.
2026-04-24T22:16:41.500132702Z ERROR: No running kube-apiserver pods found
2026-04-24T22:16:41.522571729Z Wrote inspect data to must-gather.
2026-04-24T22:16:41.559116203Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T22:16:41.747074570Z Gathering data for ns/openshift-monitoring...
2026-04-24T22:16:41.774325763Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:16:41.852741808Z Wrote inspect data to must-gather.
2026-04-24T22:16:41.897929822Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T22:16:41.929213040Z Wrote inspect data to must-gather.
2026-04-24T22:16:41.997336028Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.039869135Z No resources found
2026-04-24T22:16:42.056757092Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.100584122Z tar: Removing leading `/' from member names
2026-04-24T22:16:42.107068466Z INFO: HAProxy config collection complete.
2026-04-24T22:16:42.119980989Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T22:16:42.146536550Z INFO: 'previous-logs' folder not found on ip-10-0-133-209.ec2.internal, skipping...
2026-04-24T22:16:42.165616331Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.354290143Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T22:16:42.354880963Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.359980365Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-24T22:16:42.411297479Z INFO: 'previous-logs' folder not found on ip-10-0-133-48.ec2.internal, skipping...
2026-04-24T22:16:42.431759357Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.596007101Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.692296765Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.692472055Z INFO: 'previous-logs' folder not found on ip-10-0-136-201.ec2.internal, skipping...
2026-04-24T22:16:42.692504846Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T22:16:42.692526318Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T22:16:42.759453092Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T22:16:42.767486144Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T22:16:42.947468921Z Wrote inspect data to must-gather.
2026-04-24T22:16:42.990626353Z Wrote inspect data to must-gather.
2026-04-24T22:16:43.200216706Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T22:16:43.228975826Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:16:43.240262058Z Wrote inspect data to must-gather.
2026-04-24T22:16:43.266820157Z INFO: INTERCONNECT MODE
2026-04-24T22:16:43.266855074Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T22:16:43.431109926Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T22:16:43.445635045Z INFO: Gathering OVN_Northbound from ovnkube-node-9jw7k...
2026-04-24T22:16:43.445984161Z INFO: Gathering OVN_Northbound from ovnkube-node-vz6wr...
2026-04-24T22:16:43.446276677Z INFO: Gathering OVN_Northbound from ovnkube-node-xf75n...
2026-04-24T22:16:43.446579769Z INFO: Gathering OVN_Southbound from ovnkube-node-9jw7k...
2026-04-24T22:16:43.446855547Z INFO: Gathering OVN_Southbound from ovnkube-node-vz6wr...
2026-04-24T22:16:43.447221301Z INFO: Gathering OVN_Southbound from ovnkube-node-xf75n...
2026-04-24T22:16:43.467245927Z Wrote inspect data to must-gather.
2026-04-24T22:16:43.745010358Z tar: Removing leading `/' from member names
2026-04-24T22:16:43.759177300Z tar: Removing leading `/' from member names
2026-04-24T22:16:43.759177300Z tar: Removing leading `/' from member names
2026-04-24T22:16:43.790316378Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T22:16:43.818958598Z tar: Removing leading `/' from member names
2026-04-24T22:16:43.832054152Z tar: Removing leading `/' from member names
2026-04-24T22:16:43.835472661Z W0424 22:16:43.835335    1273 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-24T22:16:43.843362955Z Wrote inspect data to must-gather.
2026-04-24T22:16:43.855482771Z tar: Removing leading `/' from member names
2026-04-24T22:16:43.963881812Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:16:44.051466147Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T22:16:44.171763052Z Gathering data for ns/openshift-network-console...
2026-04-24T22:16:44.182212330Z Wrote inspect data to must-gather.
2026-04-24T22:16:44.356016926Z INFO: Waiting for network log collection to complete ...
2026-04-24T22:16:44.356069468Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T22:16:44.356564197Z INFO: Copying ovnk databases complete.
2026-04-24T22:16:44.358188218Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-24T22:16:44.360280149Z ovnk_database_store/
2026-04-24T22:16:44.360280149Z ovnk_database_store/ovnkube-node-xf75n_sbdb
2026-04-24T22:16:44.387086569Z ovnk_database_store/ovnkube-node-xf75n_nbdb
2026-04-24T22:16:44.398773632Z ovnk_database_store/ovnkube-node-vz6wr_sbdb
2026-04-24T22:16:44.428017763Z ovnk_database_store/ovnkube-node-9jw7k_nbdb
2026-04-24T22:16:44.437866106Z ovnk_database_store/ovnkube-node-vz6wr_nbdb
2026-04-24T22:16:44.450755314Z ovnk_database_store/ovnkube-node-9jw7k_sbdb
2026-04-24T22:16:44.476131279Z INFO: Network log collection complete.
2026-04-24T22:16:44.483585760Z INFO: Getting status from alertmanager-main-0
2026-04-24T22:16:44.518621662Z Wrote inspect data to must-gather.
2026-04-24T22:16:44.630614763Z Gathering data for ns/openshift-console-operator...
2026-04-24T22:16:44.807874539Z Wrote inspect data to must-gather.
2026-04-24T22:16:45.008621747Z Gathering data for ns/openshift-console...
2026-04-24T22:16:45.069671037Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T22:16:45.071685080Z INFO: Fallback to identify the container image from release info
2026-04-24T22:16:45.470165484Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T22:16:45.600281928Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T22:16:45.726264636Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T22:16:45.854827060Z Gathering data for ns/openshift-dns-operator...
2026-04-24T22:16:45.867776185Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T22:16:46.160509796Z Gathering data for ns/openshift-dns...
2026-04-24T22:16:46.755718946Z Gathering data for ns/openshift-image-registry...
2026-04-24T22:16:46.999991749Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T22:16:47.262907327Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T22:16:47.364948969Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-24T22:16:47.579025483Z Gathering data for ns/openshift-ingress...
2026-04-24T22:16:47.905025394Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T22:16:48.124266076Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T22:16:48.285233497Z Gathering data for ns/openshift-insights...
2026-04-24T22:16:49.242441606Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T22:16:50.373871877Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T22:16:50.385138214Z Gathering data for ns/kserve...
2026-04-24T22:16:51.488709358Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T22:16:52.014040340Z Gathering data for ns/openshift-config...
2026-04-24T22:16:52.288200344Z Gathering data for ns/openshift-config-managed...
2026-04-24T22:16:52.369631153Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-24T22:16:52.611178899Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T22:16:52.723749319Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T22:16:53.031145638Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T22:16:53.299181574Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T22:16:53.622068427Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T22:16:53.750051535Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T22:16:53.949281585Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T22:16:54.033643226Z Collecting performance related data for node ip-10-0-133-209.ec2.internal
2026-04-24T22:16:54.040383422Z Collecting performance related data for node ip-10-0-133-48.ec2.internal
2026-04-24T22:16:54.047828463Z Collecting performance related data for node ip-10-0-136-201.ec2.internal
2026-04-24T22:16:54.260455709Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T22:16:54.623990337Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T22:16:54.999144268Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T22:16:55.422034607Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T22:16:55.973093739Z Gathering data for ns/openshift-multus...
2026-04-24T22:16:57.374233596Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-24T22:16:57.651908981Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T22:16:59.357351008Z Gathering data for ns/openshift-host-network...
2026-04-24T22:16:59.681724657Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T22:17:00.212749741Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T22:17:00.689558070Z Gathering data for ns/openshift-network-operator...
2026-04-24T22:17:01.058236449Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T22:17:01.355807673Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T22:17:01.816402112Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T22:17:02.107381088Z Gathering data for ns/openshift-apiserver...
2026-04-24T22:17:02.379818084Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-04-24T22:17:02.570220077Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T22:17:02.693466033Z Collecting kubelet logs for node ip-10-0-133-209.ec2.internal
2026-04-24T22:17:02.695303630Z Collecting kubelet logs for node ip-10-0-133-48.ec2.internal
2026-04-24T22:17:02.696990043Z Collecting kubelet logs for node ip-10-0-136-201.ec2.internal
2026-04-24T22:17:02.924037634Z Gathering data for ns/openshift-controller-manager...
2026-04-24T22:17:03.103298304Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T22:17:03.113248329Z INFO: Node performance data collection complete.
2026-04-24T22:17:03.221192439Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T22:17:03.883235581Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T22:17:04.200284891Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T22:17:04.497900799Z Gathering data for ns/openshift-service-ca...
2026-04-24T22:17:04.857740023Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T22:17:05.736818639Z Wrote inspect data to must-gather.
2026-04-24T22:17:05.736858553Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:17:05.736858553Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T22:17:05.864963188Z error: the server doesn't have a resource type "clusters"
2026-04-24T22:17:05.985487492Z Caches written to disk