2026-04-24T19:28:59.657998647Z [disk usage checker] Started
2026-04-24T19:28:59.661313594Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-24T19:28:59.855923194Z Gathering data for ns/openshift-cluster-version...
2026-04-24T19:28:59.981843265Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T19:29:00.071216828Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T19:29:00.151149870Z Gathering data for ns/default...
2026-04-24T19:29:00.434750199Z Gathering data for ns/openshift...
2026-04-24T19:29:00.838587258Z Gathering data for ns/kube-system...
2026-04-24T19:29:01.500365561Z Gathering data for ns/openshift-etcd...
2026-04-24T19:29:01.831035029Z Wrote inspect data to must-gather.
2026-04-24T19:29:01.831070159Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T19:29:01.831070159Z     namespaces "assisted-installer" not found
2026-04-24T19:29:02.074790800Z Waiting on subprocesses to finish execution.
2026-04-24T19:29:02.082301893Z INFO: Collecting host service logs for crio
2026-04-24T19:29:02.082643742Z INFO: Collecting host service logs for kubelet
2026-04-24T19:29:02.082984218Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T19:29:02.083347716Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T19:29:02.083686168Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T19:29:02.083862944Z INFO: Gathering HAProxy config files
2026-04-24T19:29:02.084025441Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T19:29:02.084390377Z INFO: Collecting host service logs for NetworkManager
2026-04-24T19:29:02.084724858Z INFO: Collecting host service logs for openvswitch
2026-04-24T19:29:02.085117671Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T19:29:02.085562747Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T19:29:02.085891530Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T19:29:02.086318873Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T19:29:02.088332207Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T19:29:02.095067168Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T19:29:02.112208273Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T19:29:02.221925542Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T19:29:02.537715784Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T19:29:02.911338792Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T19:29:03.197529071Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T19:29:03.224966982Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T19:29:03.267392969Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T19:29:03.348718539Z No resources found in openshift-etcd namespace.
2026-04-24T19:29:03.350729015Z Wrote inspect data to must-gather.
2026-04-24T19:29:03.396318474Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-bzksz
2026-04-24T19:29:03.396432778Z insights-runtime-extractor-6f8xt
2026-04-24T19:29:03.396456464Z insights-runtime-extractor-6vdfs
2026-04-24T19:29:03.396477436Z insights-runtime-extractor-tt4dv
2026-04-24T19:29:03.435656028Z No resources found
2026-04-24T19:29:03.560786621Z ERROR: No running kube-apiserver pods found
2026-04-24T19:29:03.583399141Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T19:29:03.583662357Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T19:29:03.606682672Z INFO: Worker host service log collection to complete.
2026-04-24T19:29:03.625689300Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T19:29:03.644778393Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T19:29:03.651153859Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T19:29:03.882057517Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T19:29:03.882057517Z See 'oc rsync -h' for help and examples
2026-04-24T19:29:03.982367149Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.064836456Z No resources found
2026-04-24T19:29:04.205173222Z INFO: 'previous-logs' folder not found on ip-10-0-137-23.ec2.internal, skipping...
2026-04-24T19:29:04.247537536Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T19:29:04.285318350Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.396135207Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.429568105Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T19:29:04.468838195Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.505658656Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.507144893Z tar: Removing leading `/' from member names
2026-04-24T19:29:04.511677374Z INFO: HAProxy config collection complete.
2026-04-24T19:29:04.542487068Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.610520625Z Gathering data for ns/openshift-monitoring...
2026-04-24T19:29:04.668263137Z INFO: 'previous-logs' folder not found on ip-10-0-138-142.ec2.internal, skipping...
2026-04-24T19:29:04.672339096Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-24T19:29:04.725346733Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T19:29:04.742892338Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.766904084Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T19:29:04.964440540Z Wrote inspect data to must-gather.
2026-04-24T19:29:04.965901468Z INFO: 'previous-logs' folder not found on ip-10-0-138-52.ec2.internal, skipping...
2026-04-24T19:29:04.965941173Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T19:29:04.965972011Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T19:29:04.999150746Z Wrote inspect data to must-gather.
2026-04-24T19:29:05.148671666Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T19:29:05.203883246Z Wrote inspect data to must-gather.
2026-04-24T19:29:05.241219327Z Wrote inspect data to must-gather.
2026-04-24T19:29:05.348443254Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T19:29:05.521602034Z INFO: INTERCONNECT MODE
2026-04-24T19:29:05.521636201Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T19:29:05.533491110Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T19:29:05.564719022Z Wrote inspect data to must-gather.
2026-04-24T19:29:05.681676162Z INFO: Gathering OVN_Northbound from ovnkube-node-2thj7...
2026-04-24T19:29:05.681884585Z INFO: Gathering OVN_Northbound from ovnkube-node-5frl6...
2026-04-24T19:29:05.682069340Z INFO: Gathering OVN_Northbound from ovnkube-node-b2ftq...
2026-04-24T19:29:05.682348684Z INFO: Gathering OVN_Southbound from ovnkube-node-2thj7...
2026-04-24T19:29:05.682597470Z INFO: Gathering OVN_Southbound from ovnkube-node-5frl6...
2026-04-24T19:29:05.682843563Z INFO: Gathering OVN_Southbound from ovnkube-node-b2ftq...
2026-04-24T19:29:05.767975980Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T19:29:05.838211783Z Wrote inspect data to must-gather.
2026-04-24T19:29:05.985607968Z tar: Removing leading `/' from member names
2026-04-24T19:29:05.987745033Z tar: Removing leading `/' from member names
2026-04-24T19:29:05.995183629Z tar: Removing leading `/' from member names
2026-04-24T19:29:06.006887821Z tar: Removing leading `/' from member names
2026-04-24T19:29:06.011792095Z tar: Removing leading `/' from member names
2026-04-24T19:29:06.017069012Z tar: Removing leading `/' from member names
2026-04-24T19:29:06.105214016Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T19:29:06.155311834Z Wrote inspect data to must-gather.
2026-04-24T19:29:06.167143208Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T19:29:06.403766447Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T19:29:06.424506619Z Wrote inspect data to must-gather.
2026-04-24T19:29:06.492942031Z INFO: Waiting for network log collection to complete ...
2026-04-24T19:29:06.492997494Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T19:29:06.493554487Z INFO: Copying ovnk databases complete.
2026-04-24T19:29:06.495119628Z 3.3M	must-gather/network_logs/ovnk_database_store
2026-04-24T19:29:06.497324788Z ovnk_database_store/
2026-04-24T19:29:06.497324788Z ovnk_database_store/ovnkube-node-b2ftq_nbdb
2026-04-24T19:29:06.505743051Z ovnk_database_store/ovnkube-node-2thj7_sbdb
2026-04-24T19:29:06.533908036Z ovnk_database_store/ovnkube-node-5frl6_sbdb
2026-04-24T19:29:06.553677481Z ovnk_database_store/ovnkube-node-b2ftq_sbdb
2026-04-24T19:29:06.570861688Z ovnk_database_store/ovnkube-node-5frl6_nbdb
2026-04-24T19:29:06.578824515Z ovnk_database_store/ovnkube-node-2thj7_nbdb
2026-04-24T19:29:06.590655704Z INFO: Network log collection complete.
2026-04-24T19:29:06.636708977Z W0424 19:29:06.636529    1358 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-24T19:29:06.644583672Z Wrote inspect data to must-gather.
2026-04-24T19:29:06.714906461Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T19:29:06.784616248Z INFO: Getting status from alertmanager-main-0
2026-04-24T19:29:06.895240447Z Gathering data for ns/openshift-network-console...
2026-04-24T19:29:06.948287581Z Wrote inspect data to must-gather.
2026-04-24T19:29:07.237856968Z Wrote inspect data to must-gather.
2026-04-24T19:29:07.301039682Z Gathering data for ns/openshift-console-operator...
2026-04-24T19:29:07.501410741Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T19:29:07.503568221Z INFO: Fallback to identify the container image from release info
2026-04-24T19:29:07.662163947Z Gathering data for ns/openshift-console...
2026-04-24T19:29:08.071096025Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T19:29:08.082682908Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T19:29:08.208433182Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T19:29:08.339118958Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T19:29:08.456949013Z Gathering data for ns/openshift-dns-operator...
2026-04-24T19:29:08.790610253Z Gathering data for ns/openshift-dns...
2026-04-24T19:29:09.427949225Z Gathering data for ns/openshift-image-registry...
2026-04-24T19:29:09.472996826Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T19:29:09.677254566Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-24T19:29:09.882488812Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T19:29:10.242029832Z Gathering data for ns/openshift-ingress...
2026-04-24T19:29:10.570851285Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T19:29:10.603860568Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T19:29:10.931834635Z Gathering data for ns/openshift-insights...
2026-04-24T19:29:11.716591846Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T19:29:12.841073511Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T19:29:13.063712297Z Gathering data for ns/kserve...
2026-04-24T19:29:13.952918630Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T19:29:14.462796374Z Gathering data for ns/openshift-config...
2026-04-24T19:29:14.683308132Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-24T19:29:14.758159721Z Gathering data for ns/openshift-config-managed...
2026-04-24T19:29:15.074490155Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T19:29:15.210205100Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T19:29:15.454688406Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T19:29:15.701862411Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T19:29:15.967010213Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T19:29:16.206490846Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T19:29:16.267985784Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T19:29:16.495474721Z Collecting performance related data for node ip-10-0-138-142.ec2.internal
2026-04-24T19:29:16.503658844Z Collecting performance related data for node ip-10-0-138-52.ec2.internal
2026-04-24T19:29:16.512998371Z Collecting performance related data for node ip-10-0-137-23.ec2.internal
2026-04-24T19:29:16.551422709Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T19:29:16.920391193Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T19:29:17.364601050Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T19:29:17.741340289Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T19:29:18.328217005Z Gathering data for ns/openshift-multus...
2026-04-24T19:29:19.689748879Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-24T19:29:19.719951247Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T19:29:21.149804585Z Gathering data for ns/openshift-host-network...
2026-04-24T19:29:21.451286651Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T19:29:21.955909520Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T19:29:22.339138652Z Gathering data for ns/openshift-network-operator...
2026-04-24T19:29:22.711860431Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T19:29:22.961074382Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T19:29:23.388870310Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T19:29:23.688640927Z Gathering data for ns/openshift-apiserver...
2026-04-24T19:29:24.055720751Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T19:29:24.371654317Z Gathering data for ns/openshift-controller-manager...
2026-04-24T19:29:24.624740946Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T19:29:24.694838593Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-04-24T19:29:24.846546186Z Collecting kubelet logs for node ip-10-0-138-142.ec2.internal
2026-04-24T19:29:24.849332013Z Collecting kubelet logs for node ip-10-0-138-52.ec2.internal
2026-04-24T19:29:24.851553696Z Collecting kubelet logs for node ip-10-0-137-23.ec2.internal
2026-04-24T19:29:25.239053610Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T19:29:25.243319331Z INFO: Node performance data collection complete.
2026-04-24T19:29:25.347124292Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T19:29:25.650222921Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T19:29:25.988754449Z Gathering data for ns/openshift-service-ca...
2026-04-24T19:29:26.328898082Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T19:29:27.149193668Z Wrote inspect data to must-gather.
2026-04-24T19:29:27.149242742Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T19:29:27.149242742Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T19:29:27.279658136Z error: the server doesn't have a resource type "clusters"
2026-04-24T19:29:27.427390376Z Caches written to disk