2026-04-24T15:12:58.632945906Z [disk usage checker] Started
2026-04-24T15:12:58.636407100Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T15:12:58.839647612Z Gathering data for ns/openshift-cluster-version...
2026-04-24T15:12:58.981867748Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T15:12:59.066294733Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T15:12:59.148266306Z Gathering data for ns/default...
2026-04-24T15:12:59.427510037Z Gathering data for ns/openshift...
2026-04-24T15:12:59.828551683Z Gathering data for ns/kube-system...
2026-04-24T15:13:00.477336895Z Gathering data for ns/openshift-etcd...
2026-04-24T15:13:00.801412058Z Wrote inspect data to must-gather.
2026-04-24T15:13:00.801446156Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T15:13:00.801446156Z     namespaces "assisted-installer" not found
2026-04-24T15:13:01.099035648Z Waiting on subprocesses to finish execution.
2026-04-24T15:13:01.103894715Z INFO: Gathering HAProxy config files
2026-04-24T15:13:01.104831303Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T15:13:01.109854941Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T15:13:01.112792917Z INFO: Collecting host service logs for crio
2026-04-24T15:13:01.113127658Z INFO: Collecting host service logs for kubelet
2026-04-24T15:13:01.113455698Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T15:13:01.113834497Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T15:13:01.114264329Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T15:13:01.114701745Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T15:13:01.115021415Z INFO: Collecting host service logs for NetworkManager
2026-04-24T15:13:01.115317648Z INFO: Collecting host service logs for openvswitch
2026-04-24T15:13:01.115648623Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T15:13:01.116007839Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T15:13:01.116342469Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T15:13:01.116768167Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T15:13:01.235073934Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T15:13:01.295924008Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T15:13:01.582283150Z No resources found in openshift-etcd namespace.
2026-04-24T15:13:01.808902378Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T15:13:01.961062923Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T15:13:02.040214345Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T15:13:02.170507974Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T15:13:02.261466765Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T15:13:02.520172443Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T15:13:02.590053101Z No resources found
2026-04-24T15:13:02.664679152Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T15:13:02.817061430Z Wrote inspect data to must-gather.
2026-04-24T15:13:02.956649185Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T15:13:03.118329402Z Wrote inspect data to must-gather.
2026-04-24T15:13:03.164241071Z Wrote inspect data to must-gather.
2026-04-24T15:13:03.199174822Z ERROR: No running kube-apiserver pods found
2026-04-24T15:13:03.218901674Z Wrote inspect data to must-gather.
2026-04-24T15:13:03.225078253Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T15:13:03.225489962Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T15:13:03.236215322Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-hp8qf
2026-04-24T15:13:03.236262328Z insights-runtime-extractor-9z6hz
2026-04-24T15:13:03.236284338Z insights-runtime-extractor-d5lpl
2026-04-24T15:13:03.236303563Z insights-runtime-extractor-q4tm5
2026-04-24T15:13:03.332724684Z No resources found
2026-04-24T15:13:03.335980047Z INFO: Worker host service log collection to complete.
2026-04-24T15:13:03.370137945Z INFO: 'previous-logs' folder not found on ip-10-0-129-34.ec2.internal, skipping...
2026-04-24T15:13:03.411011527Z Gathering data for ns/openshift-monitoring...
2026-04-24T15:13:03.421888175Z Wrote inspect data to must-gather.
2026-04-24T15:13:03.458249204Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T15:13:03.458249204Z See 'oc rsync -h' for help and examples
2026-04-24T15:13:03.502855013Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T15:13:03.512319851Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T15:13:03.525419363Z Wrote inspect data to must-gather.
2026-04-24T15:13:03.646953890Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T15:13:03.726522902Z INFO: 'previous-logs' folder not found on ip-10-0-129-77.ec2.internal, skipping...
2026-04-24T15:13:03.741713668Z Wrote inspect data to must-gather.
2026-04-24T15:13:03.754667229Z tar: Removing leading `/' from member names
2026-04-24T15:13:03.761521294Z INFO: HAProxy config collection complete.
2026-04-24T15:13:03.789916863Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T15:13:03.790186546Z Wrote inspect data to must-gather.
2026-04-24T15:13:03.964309091Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T15:13:04.016322165Z Wrote inspect data to must-gather.
2026-04-24T15:13:04.020143317Z INFO: 'previous-logs' folder not found on ip-10-0-131-216.ec2.internal, skipping...
2026-04-24T15:13:04.020177154Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T15:13:04.020204989Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T15:13:04.192681441Z Wrote inspect data to must-gather.
2026-04-24T15:13:04.224441808Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T15:13:04.264906096Z Wrote inspect data to must-gather.
2026-04-24T15:13:04.410161099Z Wrote inspect data to must-gather.
2026-04-24T15:13:04.557247856Z INFO: INTERCONNECT MODE
2026-04-24T15:13:04.557277985Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T15:13:04.559566399Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T15:13:04.623238479Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T15:13:04.759495878Z INFO: Gathering OVN_Northbound from ovnkube-node-dz989...
2026-04-24T15:13:04.759741852Z INFO: Gathering OVN_Northbound from ovnkube-node-pbjt7...
2026-04-24T15:13:04.759914877Z INFO: Gathering OVN_Northbound from ovnkube-node-wbvmc...
2026-04-24T15:13:04.760162078Z INFO: Gathering OVN_Southbound from ovnkube-node-dz989...
2026-04-24T15:13:04.760468905Z INFO: Gathering OVN_Southbound from ovnkube-node-pbjt7...
2026-04-24T15:13:04.760812742Z INFO: Gathering OVN_Southbound from ovnkube-node-wbvmc...
2026-04-24T15:13:04.824476684Z Wrote inspect data to must-gather.
2026-04-24T15:13:04.933145247Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T15:13:04.946848809Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T15:13:05.034456761Z tar: Removing leading `/' from member names
2026-04-24T15:13:05.040822284Z tar: Removing leading `/' from member names
2026-04-24T15:13:05.061763455Z tar: Removing leading `/' from member names
2026-04-24T15:13:05.082125676Z tar: Removing leading `/' from member names
2026-04-24T15:13:05.112109894Z tar: Removing leading `/' from member names
2026-04-24T15:13:05.112829407Z tar: Removing leading `/' from member names
2026-04-24T15:13:05.153388384Z Wrote inspect data to must-gather.
2026-04-24T15:13:05.253105254Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T15:13:05.423027091Z Wrote inspect data to must-gather.
2026-04-24T15:13:05.532893222Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T15:13:05.583863998Z INFO: Waiting for network log collection to complete ...
2026-04-24T15:13:05.583900914Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T15:13:05.584410492Z INFO: Copying ovnk databases complete.
2026-04-24T15:13:05.586529731Z 3.9M	must-gather/network_logs/ovnk_database_store
2026-04-24T15:13:05.589098698Z ovnk_database_store/
2026-04-24T15:13:05.589098698Z ovnk_database_store/ovnkube-node-dz989_sbdb
2026-04-24T15:13:05.614370222Z ovnk_database_store/ovnkube-node-wbvmc_nbdb
2026-04-24T15:13:05.622636524Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T15:13:05.625252013Z ovnk_database_store/ovnkube-node-wbvmc_sbdb
2026-04-24T15:13:05.654283226Z ovnk_database_store/ovnkube-node-dz989_nbdb
2026-04-24T15:13:05.665914789Z ovnk_database_store/ovnkube-node-pbjt7_sbdb
2026-04-24T15:13:05.687689622Z ovnk_database_store/ovnkube-node-pbjt7_nbdb
2026-04-24T15:13:05.701457336Z INFO: Network log collection complete.
2026-04-24T15:13:05.738911132Z Wrote inspect data to must-gather.
2026-04-24T15:13:05.806736981Z Gathering data for ns/openshift-network-console...
2026-04-24T15:13:05.964446246Z INFO: Getting status from alertmanager-main-0
2026-04-24T15:13:06.047907265Z Wrote inspect data to must-gather.
2026-04-24T15:13:06.242788842Z Gathering data for ns/openshift-console-operator...
2026-04-24T15:13:06.351448114Z Wrote inspect data to must-gather.
2026-04-24T15:13:06.627326190Z Gathering data for ns/openshift-console...
2026-04-24T15:13:06.651210375Z Wrote inspect data to must-gather.
2026-04-24T15:13:06.915869737Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T15:13:06.918788614Z INFO: Fallback to identify the container image from release info
2026-04-24T15:13:07.048786830Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T15:13:07.376809077Z Gathering data for ns/openshift-dns-operator...
2026-04-24T15:13:07.553166786Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T15:13:07.662034587Z Gathering data for ns/openshift-dns...
2026-04-24T15:13:07.694075257Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T15:13:07.819235444Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T15:13:08.273830276Z Gathering data for ns/openshift-image-registry...
2026-04-24T15:13:08.653184790Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T15:13:08.716949881Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T15:13:08.938014752Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T15:13:09.083212066Z Gathering data for ns/openshift-ingress...
2026-04-24T15:13:09.390697029Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T15:13:09.751087938Z Gathering data for ns/openshift-insights...
2026-04-24T15:13:10.074028934Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T15:13:11.195988709Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T15:13:11.837711324Z Gathering data for ns/kserve...
2026-04-24T15:13:12.328706030Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T15:13:13.417925474Z Gathering data for ns/openshift-config...
2026-04-24T15:13:13.458790307Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T15:13:13.659069574Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T15:13:13.673450881Z Gathering data for ns/openshift-config-managed...
2026-04-24T15:13:14.070475440Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T15:13:14.319527293Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T15:13:14.586753684Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T15:13:14.600648305Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T15:13:14.863476819Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T15:13:15.176097630Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T15:13:15.421858867Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T15:13:15.742156040Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T15:13:15.754528645Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T15:13:16.022018815Z Collecting performance related data for node ip-10-0-129-77.ec2.internal
2026-04-24T15:13:16.030027306Z Collecting performance related data for node ip-10-0-129-34.ec2.internal
2026-04-24T15:13:16.040030637Z Collecting performance related data for node ip-10-0-131-216.ec2.internal
2026-04-24T15:13:16.113397623Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T15:13:16.489217796Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T15:13:17.182987916Z Gathering data for ns/openshift-multus...
2026-04-24T15:13:18.665078568Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T15:13:18.701654379Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T15:13:20.203093459Z Gathering data for ns/openshift-host-network...
2026-04-24T15:13:20.507765122Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T15:13:21.050197840Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T15:13:21.457055579Z Gathering data for ns/openshift-network-operator...
2026-04-24T15:13:21.830759654Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T15:13:22.097445700Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T15:13:22.538888066Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T15:13:22.867731325Z Gathering data for ns/openshift-apiserver...
2026-04-24T15:13:23.262697678Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T15:13:23.559454622Z Gathering data for ns/openshift-controller-manager...
2026-04-24T15:13:23.670443801Z [disk usage checker] Volume usage percentage: current = 12 ; allowed = 70
2026-04-24T15:13:23.837351004Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T15:13:24.190011070Z Collecting kubelet logs for node ip-10-0-129-77.ec2.internal
2026-04-24T15:13:24.191884162Z Collecting kubelet logs for node ip-10-0-129-34.ec2.internal
2026-04-24T15:13:24.193702744Z Collecting kubelet logs for node ip-10-0-131-216.ec2.internal
2026-04-24T15:13:24.514523632Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T15:13:24.549274190Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T15:13:24.554695296Z INFO: Node performance data collection complete.
2026-04-24T15:13:24.846074955Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T15:13:25.160479040Z Gathering data for ns/openshift-service-ca...
2026-04-24T15:13:25.503903845Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T15:13:26.352549400Z Wrote inspect data to must-gather.
2026-04-24T15:13:26.352599404Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T15:13:26.352599404Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T15:13:26.487910117Z error: the server doesn't have a resource type "clusters"
2026-04-24T15:13:26.657961777Z Caches written to disk