2026-04-17T20:19:44.966490347Z [disk usage checker] Started
2026-04-17T20:19:44.999660006Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T20:19:45.216473928Z Gathering data for ns/openshift-cluster-version...
2026-04-17T20:19:45.374311749Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T20:19:45.507234856Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T20:19:45.615717184Z Gathering data for ns/default...
2026-04-17T20:19:45.968801316Z Gathering data for ns/openshift...
2026-04-17T20:19:46.455780229Z Gathering data for ns/kube-system...
2026-04-17T20:19:47.169019581Z Gathering data for ns/openshift-etcd...
2026-04-17T20:19:47.546046815Z Wrote inspect data to must-gather.
2026-04-17T20:19:47.546137348Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T20:19:47.546137348Z     namespaces "assisted-installer" not found
2026-04-17T20:19:47.567908485Z Waiting on subprocesses to finish execution.
2026-04-17T20:19:47.578271476Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-17T20:19:47.583170061Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-17T20:19:47.589508151Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-17T20:19:47.594672936Z INFO: Gathering HAProxy config files
2026-04-17T20:19:47.595332825Z INFO: Collecting host service logs for crio
2026-04-17T20:19:47.595712836Z INFO: Collecting host service logs for kubelet
2026-04-17T20:19:47.596028162Z INFO: Collecting host service logs for rpm-ostreed
2026-04-17T20:19:47.596683474Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-17T20:19:47.597139960Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-17T20:19:47.597525416Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-17T20:19:47.597861762Z INFO: Collecting host service logs for NetworkManager
2026-04-17T20:19:47.598192717Z INFO: Collecting host service logs for openvswitch
2026-04-17T20:19:47.598523348Z INFO: Collecting host service logs for ovs-configuration
2026-04-17T20:19:47.598873655Z INFO: Collecting host service logs for ovsdb-server
2026-04-17T20:19:47.599198034Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-17T20:19:47.599735087Z INFO: Waiting for worker host service log collection to complete ...
2026-04-17T20:19:47.620332871Z INFO: Waiting for node performance related collection to complete ...
2026-04-17T20:19:47.892829434Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-17T20:19:47.893291223Z INFO: on-disk MachineConfig config collection complete.
2026-04-17T20:19:47.969963789Z No resources found
2026-04-17T20:19:48.267081130Z No resources found in openshift-etcd namespace.
2026-04-17T20:19:48.398799843Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-17T20:19:48.776086137Z error: the server doesn't have a resource type "performanceprofile"
2026-04-17T20:19:48.795661672Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-17T20:19:48.974736547Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-17T20:19:49.270968401Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-17T20:19:49.344926988Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-674s7
2026-04-17T20:19:49.345028171Z insights-runtime-extractor-bqprb
2026-04-17T20:19:49.345077041Z insights-runtime-extractor-kjsfm
2026-04-17T20:19:49.345098874Z insights-runtime-extractor-x5wz5
2026-04-17T20:19:49.397582245Z No resources found
2026-04-17T20:19:49.474268143Z INFO: Worker host service log collection to complete.
2026-04-17T20:19:49.504051216Z INFO: "metallb-operator" not detected. Skipping.
2026-04-17T20:19:49.526209769Z Wrote inspect data to must-gather.
2026-04-17T20:19:49.735420817Z Wrote inspect data to must-gather.
2026-04-17T20:19:49.908282550Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T20:19:49.998861318Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-17T20:19:50.025317366Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-17T20:19:50.029511081Z Wrote inspect data to must-gather.
2026-04-17T20:19:50.079601190Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-17T20:19:50.079601190Z See 'oc rsync -h' for help and examples
2026-04-17T20:19:50.111489715Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T20:19:50.124105865Z ERROR: No running kube-apiserver pods found
2026-04-17T20:19:50.147941218Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-17T20:19:50.163811823Z Wrote inspect data to must-gather.
2026-04-17T20:19:50.376810921Z Wrote inspect data to must-gather.
2026-04-17T20:19:50.397687575Z Wrote inspect data to must-gather.
2026-04-17T20:19:50.402763962Z tar: Removing leading `/' from member names
2026-04-17T20:19:50.413793634Z INFO: HAProxy config collection complete.
2026-04-17T20:19:50.439620954Z INFO: 'previous-logs' folder not found on ip-10-0-130-159.ec2.internal, skipping...
2026-04-17T20:19:50.539564398Z error: the server doesn't have a resource type "machineconfigs"
2026-04-17T20:19:50.652925595Z Wrote inspect data to must-gather.
2026-04-17T20:19:50.701520816Z INFO: Getting rules from prometheus-k8s-0
2026-04-17T20:19:50.844229946Z INFO: 'previous-logs' folder not found on ip-10-0-131-172.ec2.internal, skipping...
2026-04-17T20:19:50.870040994Z Wrote inspect data to must-gather.
2026-04-17T20:19:50.983727812Z Gathering data for ns/kuadrant-system...
2026-04-17T20:19:51.028889493Z Wrote inspect data to must-gather.
2026-04-17T20:19:51.213981521Z Wrote inspect data to must-gather.
2026-04-17T20:19:51.241768628Z INFO: 'previous-logs' folder not found on ip-10-0-134-158.ec2.internal, skipping...
2026-04-17T20:19:51.241862551Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-17T20:19:51.241909133Z INFO: Machine Config Daemon termination log collection complete.
2026-04-17T20:19:51.246635870Z INFO: Getting status/config from prometheus-k8s-0
2026-04-17T20:19:51.397478381Z Wrote inspect data to must-gather.
2026-04-17T20:19:51.427386138Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-17T20:19:51.777078277Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-17T20:19:51.808846004Z Wrote inspect data to must-gather.
2026-04-17T20:19:51.845835849Z INFO: INTERCONNECT MODE
2026-04-17T20:19:51.845940281Z INFO: Gathering ovn-kubernetes DBs
2026-04-17T20:19:51.924735673Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-17T20:19:52.043819457Z INFO: Gathering OVN_Northbound from ovnkube-node-jqvld...
2026-04-17T20:19:52.044097003Z INFO: Gathering OVN_Northbound from ovnkube-node-ml4d4...
2026-04-17T20:19:52.044335760Z INFO: Gathering OVN_Northbound from ovnkube-node-z2wkz...
2026-04-17T20:19:52.044663868Z INFO: Gathering OVN_Southbound from ovnkube-node-jqvld...
2026-04-17T20:19:52.044994443Z INFO: Gathering OVN_Southbound from ovnkube-node-ml4d4...
2026-04-17T20:19:52.045262017Z INFO: Gathering OVN_Southbound from ovnkube-node-z2wkz...
2026-04-17T20:19:52.045881652Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-17T20:19:52.176226412Z Wrote inspect data to must-gather.
2026-04-17T20:19:52.449752252Z tar: Removing leading `/' from member names
2026-04-17T20:19:52.485899267Z tar: Removing leading `/' from member names
2026-04-17T20:19:52.507242516Z tar: Removing leading `/' from member names
2026-04-17T20:19:52.572750059Z tar: Removing leading `/' from member names
2026-04-17T20:19:52.628773479Z tar: Removing leading `/' from member names
2026-04-17T20:19:52.641121570Z W0417 20:19:52.641050    1254 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-17T20:19:52.651795662Z tar: Removing leading `/' from member names
2026-04-17T20:19:52.676964003Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-17T20:19:52.677161141Z Wrote inspect data to must-gather.
2026-04-17T20:19:52.689221809Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-17T20:19:52.965329294Z Gathering data for ns/openshift-monitoring...
2026-04-17T20:19:53.034861337Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-17T20:19:53.145554316Z Wrote inspect data to must-gather.
2026-04-17T20:19:53.246087688Z INFO: Waiting for network log collection to complete ...
2026-04-17T20:19:53.246127238Z INFO: Waiting for ovnk database copies to complete ...
2026-04-17T20:19:53.246553283Z INFO: Copying ovnk databases complete.
2026-04-17T20:19:53.248136091Z 4.0M	must-gather/network_logs/ovnk_database_store
2026-04-17T20:19:53.250309739Z ovnk_database_store/
2026-04-17T20:19:53.250309739Z ovnk_database_store/ovnkube-node-jqvld_nbdb
2026-04-17T20:19:53.261031703Z ovnk_database_store/ovnkube-node-ml4d4_nbdb
2026-04-17T20:19:53.270420134Z ovnk_database_store/ovnkube-node-jqvld_sbdb
2026-04-17T20:19:53.297138157Z ovnk_database_store/ovnkube-node-z2wkz_nbdb
2026-04-17T20:19:53.310299376Z ovnk_database_store/ovnkube-node-z2wkz_sbdb
2026-04-17T20:19:53.339989930Z ovnk_database_store/ovnkube-node-ml4d4_sbdb
2026-04-17T20:19:53.363539962Z INFO: Network log collection complete.
2026-04-17T20:19:53.485986598Z INFO: Getting status from alertmanager-main-0
2026-04-17T20:19:53.489680601Z Wrote inspect data to must-gather.
2026-04-17T20:19:53.836894732Z Wrote inspect data to must-gather.
2026-04-17T20:19:54.158198162Z Wrote inspect data to must-gather.
2026-04-17T20:19:54.436971651Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-17T20:19:54.440334033Z INFO: Fallback to identify the container image from release info
2026-04-17T20:19:55.085723878Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-17T20:19:55.130540100Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T20:19:55.169566931Z Gathering data for ns/openshift-network-console...
2026-04-17T20:19:55.237458434Z daemonset.apps/perf-node-gather-daemonset created
2026-04-17T20:19:55.407664107Z Waiting for performance profile collector pods to become ready: 1
2026-04-17T20:19:55.753339594Z Gathering data for ns/openshift-console-operator...
2026-04-17T20:19:56.265902595Z Gathering data for ns/openshift-console...
2026-04-17T20:19:56.533043116Z Waiting for performance profile collector pods to become ready: 2
2026-04-17T20:19:56.801079194Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-17T20:19:56.878214122Z Wrote inspect data to must-gather.
2026-04-17T20:19:57.174888157Z Gathering data for ns/openshift-dns-operator...
2026-04-17T20:19:57.482502532Z Gathering data for ns/openshift-dns...
2026-04-17T20:19:57.644614518Z Waiting for performance profile collector pods to become ready: 3
2026-04-17T20:19:58.112519368Z Gathering data for ns/openshift-image-registry...
2026-04-17T20:19:58.589987030Z Gathering data for ns/openshift-ingress-operator...
2026-04-17T20:19:58.770084864Z Waiting for performance profile collector pods to become ready: 4
2026-04-17T20:19:58.956086590Z Gathering data for ns/openshift-ingress...
2026-04-17T20:19:59.607801244Z Gathering data for ns/openshift-ingress-canary...
2026-04-17T20:19:59.892842300Z Waiting for performance profile collector pods to become ready: 5
2026-04-17T20:20:00.080701259Z Gathering data for ns/openshift-insights...
2026-04-17T20:20:00.135314146Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T20:20:01.018424094Z Waiting for performance profile collector pods to become ready: 6
2026-04-17T20:20:02.062597726Z Gathering data for ns/opendatahub...
2026-04-17T20:20:02.145448749Z Waiting for performance profile collector pods to become ready: 7
2026-04-17T20:20:03.262716879Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-17T20:20:03.389934908Z Gathering data for ns/openshift-lws-operator...
2026-04-17T20:20:03.542557405Z Collecting performance related data for node ip-10-0-130-159.ec2.internal
2026-04-17T20:20:03.552562408Z Collecting performance related data for node ip-10-0-131-172.ec2.internal
2026-04-17T20:20:03.564680548Z Collecting performance related data for node ip-10-0-134-158.ec2.internal
2026-04-17T20:20:04.894636022Z Gathering data for ns/openshift-config...
2026-04-17T20:20:05.143964030Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T20:20:05.280158382Z Gathering data for ns/openshift-config-managed...
2026-04-17T20:20:05.736815135Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-17T20:20:06.090795535Z Gathering data for ns/openshift-kube-apiserver...
2026-04-17T20:20:06.433493216Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-17T20:20:06.813925519Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-17T20:20:07.149384564Z Gathering data for ns/openshift-kube-scheduler...
2026-04-17T20:20:07.486751867Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-17T20:20:07.829048063Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-17T20:20:08.203139095Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-17T20:20:08.566088993Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-17T20:20:09.193650713Z Gathering data for ns/openshift-multus...
2026-04-17T20:20:10.149380296Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T20:20:10.832889434Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-17T20:20:12.354570047Z Gathering data for ns/openshift-host-network...
2026-04-17T20:20:12.645406506Z Collecting kubelet logs for node ip-10-0-130-159.ec2.internal
2026-04-17T20:20:12.647194612Z Collecting kubelet logs for node ip-10-0-131-172.ec2.internal
2026-04-17T20:20:12.648881914Z Collecting kubelet logs for node ip-10-0-134-158.ec2.internal
2026-04-17T20:20:12.716857172Z Gathering data for ns/openshift-network-diagnostics...
2026-04-17T20:20:13.039106985Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-17T20:20:13.044496000Z INFO: Node performance data collection complete.
2026-04-17T20:20:13.377778241Z Gathering data for ns/openshift-network-node-identity...
2026-04-17T20:20:13.846818042Z Gathering data for ns/openshift-network-operator...
2026-04-17T20:20:14.269683952Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-17T20:20:14.580602553Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-17T20:20:15.077238439Z Gathering data for ns/openshift-apiserver-operator...
2026-04-17T20:20:15.156962713Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-17T20:20:15.416051895Z Gathering data for ns/openshift-apiserver...
2026-04-17T20:20:15.864483678Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-17T20:20:16.194724251Z Gathering data for ns/openshift-controller-manager...
2026-04-17T20:20:16.497281520Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-17T20:20:17.264384006Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-17T20:20:17.597380041Z Gathering data for ns/openshift-service-ca-operator...
2026-04-17T20:20:17.945553165Z Gathering data for ns/openshift-service-ca...
2026-04-17T20:20:18.387091561Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-17T20:20:19.403427206Z Wrote inspect data to must-gather.
2026-04-17T20:20:19.403475332Z error: inspection completed with the errors occurred while gathering data:
2026-04-17T20:20:19.403475332Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-17T20:20:19.539695176Z error: the server doesn't have a resource type "clusters"
2026-04-17T20:20:19.722331365Z Caches written to disk