2026-05-29T02:27:37.028832299Z [disk usage checker] Started
2026-05-29T02:27:37.032745939Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T02:27:37.225779244Z Gathering data for ns/openshift-cluster-version...
2026-05-29T02:27:37.368385403Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T02:27:37.464181384Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T02:27:37.543375882Z Gathering data for ns/default...
2026-05-29T02:27:37.830453689Z Gathering data for ns/openshift...
2026-05-29T02:27:38.240455718Z Gathering data for ns/kube-system...
2026-05-29T02:27:38.975739500Z Gathering data for ns/openshift-etcd...
2026-05-29T02:27:39.262594433Z Wrote inspect data to must-gather.
2026-05-29T02:27:39.262628518Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T02:27:39.262628518Z     namespaces "assisted-installer" not found
2026-05-29T02:27:39.277661861Z Waiting on subprocesses to finish execution.
2026-05-29T02:27:39.293446474Z INFO: Gathering HAProxy config files
2026-05-29T02:27:39.299706373Z INFO: Collecting host service logs for crio
2026-05-29T02:27:39.300092306Z INFO: Collecting host service logs for kubelet
2026-05-29T02:27:39.300446406Z INFO: Collecting host service logs for rpm-ostreed
2026-05-29T02:27:39.300802857Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-29T02:27:39.301175987Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-29T02:27:39.301528840Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-29T02:27:39.301902311Z INFO: Collecting host service logs for NetworkManager
2026-05-29T02:27:39.302284001Z INFO: Collecting host service logs for openvswitch
2026-05-29T02:27:39.302624742Z INFO: Collecting host service logs for ovs-configuration
2026-05-29T02:27:39.302986589Z INFO: Collecting host service logs for ovsdb-server
2026-05-29T02:27:39.303329524Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-29T02:27:39.303388270Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-29T02:27:39.303768311Z INFO: Waiting for worker host service log collection to complete ...
2026-05-29T02:27:39.311045882Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-29T02:27:39.313979936Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-29T02:27:39.386984682Z INFO: Waiting for node performance related collection to complete ...
2026-05-29T02:27:39.646507403Z No resources found in openshift-etcd namespace.
2026-05-29T02:27:39.736262532Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-29T02:27:39.957073649Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-29T02:27:39.957158642Z INFO: on-disk MachineConfig config collection complete.
2026-05-29T02:27:40.031348207Z INFO: "metallb-operator" not detected. Skipping.
2026-05-29T02:27:40.099171285Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-29T02:27:40.122151632Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-29T02:27:40.343708961Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-29T02:27:40.377375035Z Wrote inspect data to must-gather.
2026-05-29T02:27:40.404339251Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-wkc46
2026-05-29T02:27:40.404394295Z insights-runtime-extractor-f7hft
2026-05-29T02:27:40.404426332Z insights-runtime-extractor-gqztr
2026-05-29T02:27:40.404445325Z insights-runtime-extractor-kh8m5
2026-05-29T02:27:40.495119481Z No resources found
2026-05-29T02:27:40.520505855Z INFO: Worker host service log collection to complete.
2026-05-29T02:27:40.612485420Z error: the server doesn't have a resource type "performanceprofile"
2026-05-29T02:27:40.643864571Z Wrote inspect data to must-gather.
2026-05-29T02:27:40.661198931Z ERROR: No running kube-apiserver pods found
2026-05-29T02:27:40.723633625Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-29T02:27:40.723633625Z See 'oc rsync -h' for help and examples
2026-05-29T02:27:40.854043486Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-29T02:27:40.901677076Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-29T02:27:41.115995237Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.118257244Z Gathering data for ns/openshift-monitoring...
2026-05-29T02:27:41.197370622Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.254239670Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.354544269Z No resources found
2026-05-29T02:27:41.377894966Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T02:27:41.407130633Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.460566964Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-29T02:27:41.488097557Z INFO: 'previous-logs' folder not found on ip-10-0-129-62.ec2.internal, skipping...
2026-05-29T02:27:41.493492671Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.517130604Z tar: Removing leading `/' from member names
2026-05-29T02:27:41.521484601Z INFO: HAProxy config collection complete.
2026-05-29T02:27:41.601613882Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.649239973Z INFO: Getting rules from prometheus-k8s-0
2026-05-29T02:27:41.670776634Z error: the server doesn't have a resource type "machineconfigs"
2026-05-29T02:27:41.748971800Z INFO: 'previous-logs' folder not found on ip-10-0-130-154.ec2.internal, skipping...
2026-05-29T02:27:41.861260582Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.883403324Z Wrote inspect data to must-gather.
2026-05-29T02:27:41.998012786Z INFO: 'previous-logs' folder not found on ip-10-0-141-250.ec2.internal, skipping...
2026-05-29T02:27:41.998055385Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-29T02:27:41.998066145Z INFO: Machine Config Daemon termination log collection complete.
2026-05-29T02:27:42.038376512Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T02:27:42.058098896Z INFO: Getting status/config from prometheus-k8s-0
2026-05-29T02:27:42.094924612Z Wrote inspect data to must-gather.
2026-05-29T02:27:42.103843928Z Wrote inspect data to must-gather.
2026-05-29T02:27:42.235364239Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-29T02:27:42.363104130Z INFO: INTERCONNECT MODE
2026-05-29T02:27:42.363144425Z INFO: Gathering ovn-kubernetes DBs
2026-05-29T02:27:42.419128396Z Wrote inspect data to must-gather.
2026-05-29T02:27:42.465181827Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-29T02:27:42.513022440Z INFO: Gathering OVN_Northbound from ovnkube-node-7v5zh...
2026-05-29T02:27:42.513269665Z INFO: Gathering OVN_Northbound from ovnkube-node-8484b...
2026-05-29T02:27:42.513515204Z INFO: Gathering OVN_Northbound from ovnkube-node-pktj7...
2026-05-29T02:27:42.513808984Z INFO: Gathering OVN_Southbound from ovnkube-node-7v5zh...
2026-05-29T02:27:42.514081751Z INFO: Gathering OVN_Southbound from ovnkube-node-8484b...
2026-05-29T02:27:42.514336785Z INFO: Gathering OVN_Southbound from ovnkube-node-pktj7...
2026-05-29T02:27:42.691345385Z Wrote inspect data to must-gather.
2026-05-29T02:27:42.765480048Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-29T02:27:42.803294219Z tar: Removing leading `/' from member names
2026-05-29T02:27:42.813228670Z tar: Removing leading `/' from member names
2026-05-29T02:27:42.833656882Z tar: Removing leading `/' from member names
2026-05-29T02:27:42.840117687Z tar: Removing leading `/' from member names
2026-05-29T02:27:42.844140539Z tar: Removing leading `/' from member names
2026-05-29T02:27:42.867075178Z tar: Removing leading `/' from member names
2026-05-29T02:27:42.917405432Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-29T02:27:42.977253946Z Wrote inspect data to must-gather.
2026-05-29T02:27:43.026823493Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-29T02:27:43.220305478Z Wrote inspect data to must-gather.
2026-05-29T02:27:43.276031091Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-29T02:27:43.312479896Z INFO: Waiting for network log collection to complete ...
2026-05-29T02:27:43.312515911Z INFO: Waiting for ovnk database copies to complete ...
2026-05-29T02:27:43.312899780Z INFO: Copying ovnk databases complete.
2026-05-29T02:27:43.314455644Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-05-29T02:27:43.316498175Z ovnk_database_store/
2026-05-29T02:27:43.316498175Z ovnk_database_store/ovnkube-node-7v5zh_nbdb
2026-05-29T02:27:43.329935633Z ovnk_database_store/ovnkube-node-8484b_sbdb
2026-05-29T02:27:43.348886336Z ovnk_database_store/ovnkube-node-pktj7_nbdb
2026-05-29T02:27:43.361429178Z ovnk_database_store/ovnkube-node-pktj7_sbdb
2026-05-29T02:27:43.385539181Z ovnk_database_store/ovnkube-node-8484b_nbdb
2026-05-29T02:27:43.395974106Z ovnk_database_store/ovnkube-node-7v5zh_sbdb
2026-05-29T02:27:43.417821488Z INFO: Network log collection complete.
2026-05-29T02:27:43.420356857Z Wrote inspect data to must-gather.
2026-05-29T02:27:43.545278252Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-29T02:27:43.654278413Z INFO: Getting status from alertmanager-main-0
2026-05-29T02:27:43.682148480Z Wrote inspect data to must-gather.
2026-05-29T02:27:43.719166017Z Gathering data for ns/openshift-network-console...
2026-05-29T02:27:43.952211710Z Wrote inspect data to must-gather.
2026-05-29T02:27:44.106675145Z Gathering data for ns/openshift-console-operator...
2026-05-29T02:27:44.216087267Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-29T02:27:44.218270870Z INFO: Fallback to identify the container image from release info
2026-05-29T02:27:44.438808611Z Gathering data for ns/openshift-console...
2026-05-29T02:27:44.780382792Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-29T02:27:44.831050741Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-29T02:27:44.899338693Z daemonset.apps/perf-node-gather-daemonset created
2026-05-29T02:27:45.023170675Z Waiting for performance profile collector pods to become ready: 1
2026-05-29T02:27:45.175478138Z Gathering data for ns/openshift-dns-operator...
2026-05-29T02:27:45.441257691Z Gathering data for ns/openshift-dns...
2026-05-29T02:27:45.956690561Z Gathering data for ns/openshift-image-registry...
2026-05-29T02:27:46.145452677Z Waiting for performance profile collector pods to become ready: 2
2026-05-29T02:27:46.419051435Z Gathering data for ns/openshift-ingress-operator...
2026-05-29T02:27:46.715037470Z Gathering data for ns/openshift-ingress...
2026-05-29T02:27:47.015202211Z Gathering data for ns/openshift-ingress-canary...
2026-05-29T02:27:47.044914015Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T02:27:47.262272952Z Waiting for performance profile collector pods to become ready: 3
2026-05-29T02:27:47.426096430Z Gathering data for ns/openshift-insights...
2026-05-29T02:27:48.369734621Z Waiting for performance profile collector pods to become ready: 4
2026-05-29T02:27:49.354722418Z Gathering data for ns/openshift-config...
2026-05-29T02:27:49.485716615Z Waiting for performance profile collector pods to become ready: 5
2026-05-29T02:27:49.589911476Z Gathering data for ns/openshift-config-managed...
2026-05-29T02:27:50.002527177Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-29T02:27:50.292477805Z Gathering data for ns/openshift-kube-apiserver...
2026-05-29T02:27:50.592216592Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-29T02:27:50.605713735Z Waiting for performance profile collector pods to become ready: 6
2026-05-29T02:27:50.845067448Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-29T02:27:51.138417252Z Gathering data for ns/openshift-kube-scheduler...
2026-05-29T02:27:51.366376001Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-29T02:27:51.681098930Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-29T02:27:51.720176046Z Waiting for performance profile collector pods to become ready: 7
2026-05-29T02:27:51.964125272Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-29T02:27:52.049500512Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T02:27:52.245368383Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-29T02:27:52.680171349Z Gathering data for ns/openshift-multus...
2026-05-29T02:27:52.820537209Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-29T02:27:53.038811795Z Collecting performance related data for node ip-10-0-141-250.ec2.internal
2026-05-29T02:27:53.045752782Z Collecting performance related data for node ip-10-0-130-154.ec2.internal
2026-05-29T02:27:53.052975428Z Collecting performance related data for node ip-10-0-129-62.ec2.internal
2026-05-29T02:27:54.157939690Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-29T02:27:55.583661265Z Gathering data for ns/openshift-host-network...
2026-05-29T02:27:55.874817342Z Gathering data for ns/openshift-network-diagnostics...
2026-05-29T02:27:56.381640735Z Gathering data for ns/openshift-network-node-identity...
2026-05-29T02:27:56.743265213Z Gathering data for ns/openshift-network-operator...
2026-05-29T02:27:57.054497058Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-05-29T02:27:57.089536999Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-29T02:27:57.327596531Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-29T02:27:57.744356152Z Gathering data for ns/openshift-apiserver-operator...
2026-05-29T02:27:58.009775793Z Gathering data for ns/openshift-apiserver...
2026-05-29T02:27:58.433281580Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-29T02:27:58.751099779Z Gathering data for ns/openshift-controller-manager...
2026-05-29T02:27:59.037480780Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-29T02:27:59.635310054Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-29T02:27:59.942369234Z Gathering data for ns/openshift-service-ca-operator...
2026-05-29T02:28:00.293174404Z Gathering data for ns/openshift-service-ca...
2026-05-29T02:28:00.655731932Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-29T02:28:01.168156386Z Collecting kubelet logs for node ip-10-0-141-250.ec2.internal
2026-05-29T02:28:01.169843506Z Collecting kubelet logs for node ip-10-0-130-154.ec2.internal
2026-05-29T02:28:01.171604117Z Collecting kubelet logs for node ip-10-0-129-62.ec2.internal
2026-05-29T02:28:01.514613320Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-29T02:28:01.520758369Z INFO: Node performance data collection complete.
2026-05-29T02:28:01.536024374Z Wrote inspect data to must-gather.
2026-05-29T02:28:01.536063017Z error: inspection completed with the errors occurred while gathering data:
2026-05-29T02:28:01.536063017Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-29T02:28:01.664723195Z error: the server doesn't have a resource type "clusters"
2026-05-29T02:28:01.836400033Z Caches written to disk