2026-04-20T11:46:23.038171353Z [disk usage checker] Started
2026-04-20T11:46:23.041950592Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T11:46:23.240785305Z Gathering data for ns/openshift-cluster-version...
2026-04-20T11:46:23.382154743Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T11:46:23.459683935Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T11:46:23.546589506Z Gathering data for ns/default...
2026-04-20T11:46:23.826475193Z Gathering data for ns/openshift...
2026-04-20T11:46:24.206665086Z Gathering data for ns/kube-system...
2026-04-20T11:46:24.816021007Z Gathering data for ns/openshift-etcd...
2026-04-20T11:46:25.105247348Z Wrote inspect data to must-gather.
2026-04-20T11:46:25.105280477Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T11:46:25.105280477Z     namespaces "assisted-installer" not found
2026-04-20T11:46:25.272836700Z Waiting on subprocesses to finish execution.
2026-04-20T11:46:25.280753230Z INFO: Gathering HAProxy config files
2026-04-20T11:46:25.289655394Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-20T11:46:25.290466533Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-20T11:46:25.293163635Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-20T11:46:25.293360447Z INFO: Collecting host service logs for crio
2026-04-20T11:46:25.293666949Z INFO: Collecting host service logs for kubelet
2026-04-20T11:46:25.293988384Z INFO: Collecting host service logs for rpm-ostreed
2026-04-20T11:46:25.294291199Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-20T11:46:25.294596316Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-20T11:46:25.294973798Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-20T11:46:25.295274268Z INFO: Collecting host service logs for NetworkManager
2026-04-20T11:46:25.295579058Z INFO: Collecting host service logs for openvswitch
2026-04-20T11:46:25.295888706Z INFO: Collecting host service logs for ovs-configuration
2026-04-20T11:46:25.296184725Z INFO: Collecting host service logs for ovsdb-server
2026-04-20T11:46:25.296479217Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-20T11:46:25.296840120Z INFO: Waiting for worker host service log collection to complete ...
2026-04-20T11:46:25.304578612Z INFO: Waiting for node performance related collection to complete ...
2026-04-20T11:46:25.812052820Z No resources found in openshift-etcd namespace.
2026-04-20T11:46:25.849829168Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-20T11:46:25.923582968Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-4hfdx
2026-04-20T11:46:25.923582968Z insights-runtime-extractor-f6qb2
2026-04-20T11:46:25.923582968Z insights-runtime-extractor-t2dhh
2026-04-20T11:46:25.923582968Z insights-runtime-extractor-zps6q
2026-04-20T11:46:25.925039365Z INFO: "metallb-operator" not detected. Skipping.
2026-04-20T11:46:26.077257029Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-20T11:46:26.142587700Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-20T11:46:26.209833595Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-20T11:46:26.264905870Z ERROR: No running kube-apiserver pods found
2026-04-20T11:46:26.357074120Z Wrote inspect data to must-gather.
2026-04-20T11:46:26.429087892Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-20T11:46:26.429198129Z INFO: on-disk MachineConfig config collection complete.
2026-04-20T11:46:26.430075207Z No resources found
2026-04-20T11:46:26.500025153Z error: the server doesn't have a resource type "performanceprofile"
2026-04-20T11:46:26.669160640Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-20T11:46:26.669160640Z See 'oc rsync -h' for help and examples
2026-04-20T11:46:26.797099521Z INFO: Worker host service log collection to complete.
2026-04-20T11:46:26.803619529Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-20T11:46:27.074431839Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-20T11:46:27.087824368Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.114617007Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-20T11:46:27.228255294Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.236246987Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.267095394Z INFO: 'previous-logs' folder not found on ip-10-0-133-125.ec2.internal, skipping...
2026-04-20T11:46:27.277171459Z No resources found
2026-04-20T11:46:27.353472843Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.391537637Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T11:46:27.478369791Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.487657861Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.539063569Z tar: Removing leading `/' from member names
2026-04-20T11:46:27.544259992Z INFO: HAProxy config collection complete.
2026-04-20T11:46:27.549649849Z error: the server doesn't have a resource type "machineconfigs"
2026-04-20T11:46:27.649418841Z INFO: 'previous-logs' folder not found on ip-10-0-135-68.ec2.internal, skipping...
2026-04-20T11:46:27.655916667Z Gathering data for ns/openshift-monitoring...
2026-04-20T11:46:27.727016889Z INFO: Getting rules from prometheus-k8s-0
2026-04-20T11:46:27.741338303Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.770430906Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.900836158Z INFO: 'previous-logs' folder not found on ip-10-0-141-26.ec2.internal, skipping...
2026-04-20T11:46:27.900864911Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-20T11:46:27.900871929Z INFO: Machine Config Daemon termination log collection complete.
2026-04-20T11:46:27.964836395Z Wrote inspect data to must-gather.
2026-04-20T11:46:27.978843368Z Wrote inspect data to must-gather.
2026-04-20T11:46:28.047233225Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T11:46:28.092512355Z INFO: Getting status/config from prometheus-k8s-0
2026-04-20T11:46:28.132924684Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-20T11:46:28.185141668Z Wrote inspect data to must-gather.
2026-04-20T11:46:28.318716947Z Wrote inspect data to must-gather.
2026-04-20T11:46:28.442657831Z INFO: INTERCONNECT MODE
2026-04-20T11:46:28.442707493Z INFO: Gathering ovn-kubernetes DBs
2026-04-20T11:46:28.446307650Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-20T11:46:28.518791860Z Wrote inspect data to must-gather.
2026-04-20T11:46:28.588605680Z INFO: Gathering OVN_Northbound from ovnkube-node-8q26d...
2026-04-20T11:46:28.588840594Z INFO: Gathering OVN_Northbound from ovnkube-node-h966s...
2026-04-20T11:46:28.589076035Z INFO: Gathering OVN_Northbound from ovnkube-node-l6c62...
2026-04-20T11:46:28.589387325Z INFO: Gathering OVN_Southbound from ovnkube-node-8q26d...
2026-04-20T11:46:28.589637309Z INFO: Gathering OVN_Southbound from ovnkube-node-h966s...
2026-04-20T11:46:28.589935675Z INFO: Gathering OVN_Southbound from ovnkube-node-l6c62...
2026-04-20T11:46:28.711796161Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-20T11:46:28.804244348Z Wrote inspect data to must-gather.
2026-04-20T11:46:28.890787142Z tar: Removing leading `/' from member names
2026-04-20T11:46:28.894468054Z tar: Removing leading `/' from member names
2026-04-20T11:46:28.899653072Z tar: Removing leading `/' from member names
2026-04-20T11:46:28.899784929Z tar: Removing leading `/' from member names
2026-04-20T11:46:28.900037685Z tar: Removing leading `/' from member names
2026-04-20T11:46:28.913850647Z tar: Removing leading `/' from member names
2026-04-20T11:46:29.002790665Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-20T11:46:29.055019586Z Wrote inspect data to must-gather.
2026-04-20T11:46:29.106749017Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-20T11:46:29.238277921Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-20T11:46:29.270493656Z Wrote inspect data to must-gather.
2026-04-20T11:46:29.343441793Z INFO: Waiting for network log collection to complete ...
2026-04-20T11:46:29.343567728Z INFO: Waiting for ovnk database copies to complete ...
2026-04-20T11:46:29.344117958Z INFO: Copying ovnk databases complete.
2026-04-20T11:46:29.346201801Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-04-20T11:46:29.348907380Z ovnk_database_store/
2026-04-20T11:46:29.348907380Z ovnk_database_store/ovnkube-node-l6c62_nbdb
2026-04-20T11:46:29.356531280Z ovnk_database_store/ovnkube-node-8q26d_nbdb
2026-04-20T11:46:29.363111184Z ovnk_database_store/ovnkube-node-l6c62_sbdb
2026-04-20T11:46:29.378019172Z ovnk_database_store/ovnkube-node-8q26d_sbdb
2026-04-20T11:46:29.393512358Z ovnk_database_store/ovnkube-node-h966s_sbdb
2026-04-20T11:46:29.409612997Z ovnk_database_store/ovnkube-node-h966s_nbdb
2026-04-20T11:46:29.417372748Z INFO: Network log collection complete.
2026-04-20T11:46:29.473366048Z Wrote inspect data to must-gather.
2026-04-20T11:46:29.583881603Z INFO: Getting status from alertmanager-main-0
2026-04-20T11:46:29.653871271Z W0420 11:46:29.653811    1398 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-20T11:46:29.659044926Z Wrote inspect data to must-gather.
2026-04-20T11:46:29.695236052Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-20T11:46:29.855172499Z Gathering data for ns/openshift-network-console...
2026-04-20T11:46:29.894351809Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-20T11:46:29.897378187Z INFO: Fallback to identify the container image from release info
2026-04-20T11:46:30.230891381Z Gathering data for ns/openshift-console-operator...
2026-04-20T11:46:30.451194962Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-20T11:46:30.569877793Z daemonset.apps/perf-node-gather-daemonset created
2026-04-20T11:46:30.595558665Z Gathering data for ns/openshift-console...
2026-04-20T11:46:30.681324015Z Waiting for performance profile collector pods to become ready: 1
2026-04-20T11:46:31.035370051Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-20T11:46:31.409198254Z Gathering data for ns/openshift-dns-operator...
2026-04-20T11:46:31.657675333Z Gathering data for ns/openshift-dns...
2026-04-20T11:46:31.782575796Z Waiting for performance profile collector pods to become ready: 2
2026-04-20T11:46:32.296759923Z Gathering data for ns/openshift-image-registry...
2026-04-20T11:46:32.759283425Z Gathering data for ns/openshift-ingress-operator...
2026-04-20T11:46:32.895784635Z Waiting for performance profile collector pods to become ready: 3
2026-04-20T11:46:33.057040439Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T11:46:33.066574256Z Gathering data for ns/openshift-ingress...
2026-04-20T11:46:33.390423252Z Gathering data for ns/openshift-ingress-canary...
2026-04-20T11:46:33.737812295Z Gathering data for ns/openshift-insights...
2026-04-20T11:46:34.004466370Z Waiting for performance profile collector pods to become ready: 4
2026-04-20T11:46:35.123076862Z Waiting for performance profile collector pods to become ready: 5
2026-04-20T11:46:35.524994390Z Gathering data for ns/openshift-config...
2026-04-20T11:46:35.781605384Z Gathering data for ns/openshift-config-managed...
2026-04-20T11:46:36.151396488Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-20T11:46:36.245060548Z Waiting for performance profile collector pods to become ready: 6
2026-04-20T11:46:36.380386807Z Gathering data for ns/openshift-kube-apiserver...
2026-04-20T11:46:36.595030683Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-20T11:46:36.830467820Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-20T11:46:37.083390776Z Gathering data for ns/openshift-kube-scheduler...
2026-04-20T11:46:37.297449391Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-20T11:46:37.373857959Z Waiting for performance profile collector pods to become ready: 7
2026-04-20T11:46:37.550219407Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-20T11:46:37.817199078Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-20T11:46:38.063634490Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T11:46:38.071028691Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-20T11:46:38.490054226Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-20T11:46:38.548737847Z Gathering data for ns/openshift-multus...
2026-04-20T11:46:38.725325379Z Collecting performance related data for node ip-10-0-141-26.ec2.internal
2026-04-20T11:46:38.732367017Z Collecting performance related data for node ip-10-0-135-68.ec2.internal
2026-04-20T11:46:38.740227897Z Collecting performance related data for node ip-10-0-133-125.ec2.internal
2026-04-20T11:46:39.927132965Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-20T11:46:41.500454272Z Gathering data for ns/openshift-host-network...
2026-04-20T11:46:41.793522436Z Gathering data for ns/openshift-network-diagnostics...
2026-04-20T11:46:42.239272876Z Gathering data for ns/openshift-network-node-identity...
2026-04-20T11:46:42.582239406Z Gathering data for ns/openshift-network-operator...
2026-04-20T11:46:42.913610798Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-20T11:46:43.068435831Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-04-20T11:46:43.198861672Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-20T11:46:43.601078104Z Gathering data for ns/openshift-apiserver-operator...
2026-04-20T11:46:43.874360614Z Gathering data for ns/openshift-apiserver...
2026-04-20T11:46:44.230138495Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-20T11:46:44.480013586Z Gathering data for ns/openshift-controller-manager...
2026-04-20T11:46:44.733067144Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-20T11:46:45.339426467Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-20T11:46:45.586107630Z Gathering data for ns/openshift-service-ca-operator...
2026-04-20T11:46:45.865293739Z Gathering data for ns/openshift-service-ca...
2026-04-20T11:46:46.188023036Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-20T11:46:46.582779147Z Collecting kubelet logs for node ip-10-0-141-26.ec2.internal
2026-04-20T11:46:46.585009616Z Collecting kubelet logs for node ip-10-0-135-68.ec2.internal
2026-04-20T11:46:46.587934180Z Collecting kubelet logs for node ip-10-0-133-125.ec2.internal
2026-04-20T11:46:46.908822135Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-20T11:46:46.913307757Z INFO: Node performance data collection complete.
2026-04-20T11:46:47.062516734Z Wrote inspect data to must-gather.
2026-04-20T11:46:47.062558922Z error: inspection completed with the errors occurred while gathering data:
2026-04-20T11:46:47.062558922Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-20T11:46:47.195137383Z error: the server doesn't have a resource type "clusters"
2026-04-20T11:46:47.325990283Z Caches written to disk