2026-05-28T15:39:38.921557099Z [disk usage checker] Started
2026-05-28T15:39:38.927531684Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-28T15:39:39.096802497Z Gathering data for ns/openshift-cluster-version...
2026-05-28T15:39:39.208361934Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T15:39:39.287343660Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T15:39:39.356097148Z Gathering data for ns/default...
2026-05-28T15:39:39.610674228Z Gathering data for ns/openshift...
2026-05-28T15:39:39.976045754Z Gathering data for ns/kube-system...
2026-05-28T15:39:40.602833523Z Gathering data for ns/openshift-etcd...
2026-05-28T15:39:40.860896747Z Wrote inspect data to must-gather.
2026-05-28T15:39:40.861009386Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T15:39:40.861009386Z     namespaces "assisted-installer" not found
2026-05-28T15:39:41.116503143Z Waiting on subprocesses to finish execution.
2026-05-28T15:39:41.122235483Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-05-28T15:39:41.125193321Z INFO: Gathering HAProxy config files
2026-05-28T15:39:41.130358727Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-05-28T15:39:41.151384660Z INFO: Collecting host service logs for crio
2026-05-28T15:39:41.151384660Z INFO: Gathering machine config daemon's old logs from all nodes
2026-05-28T15:39:41.151698630Z INFO: Collecting host service logs for kubelet
2026-05-28T15:39:41.152001601Z INFO: Collecting host service logs for rpm-ostreed
2026-05-28T15:39:41.152350309Z INFO: Collecting host service logs for ostree-finalize-staged
2026-05-28T15:39:41.152645238Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-05-28T15:39:41.152934872Z INFO: Collecting host service logs for machine-config-daemon-host
2026-05-28T15:39:41.153254157Z INFO: Collecting host service logs for NetworkManager
2026-05-28T15:39:41.153545397Z INFO: Collecting host service logs for openvswitch
2026-05-28T15:39:41.153833909Z INFO: Collecting host service logs for ovs-configuration
2026-05-28T15:39:41.154161134Z INFO: Collecting host service logs for ovsdb-server
2026-05-28T15:39:41.154441285Z INFO: Collecting host service logs for ovs-vswitchd
2026-05-28T15:39:41.154792238Z INFO: Waiting for worker host service log collection to complete ...
2026-05-28T15:39:41.404507995Z INFO: Waiting for node performance related collection to complete ...
2026-05-28T15:39:41.596562747Z No resources found in openshift-etcd namespace.
2026-05-28T15:39:41.922165641Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-05-28T15:39:41.944173619Z INFO: "sriov-network-operator" not detected. Skipping.
2026-05-28T15:39:41.986237673Z INFO: Collecting Insights Archives from insights-operator-78bdfb888c-r4zls
2026-05-28T15:39:41.986237673Z insights-runtime-extractor-p7wzv
2026-05-28T15:39:41.986237673Z insights-runtime-extractor-qq7rt
2026-05-28T15:39:41.986237673Z insights-runtime-extractor-vg8zf
2026-05-28T15:39:41.991481374Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-05-28T15:39:41.991569375Z INFO: on-disk MachineConfig config collection complete.
2026-05-28T15:39:42.003248762Z INFO: "metallb-operator" not detected. Skipping.
2026-05-28T15:39:42.008475992Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-05-28T15:39:42.230246718Z Wrote inspect data to must-gather.
2026-05-28T15:39:42.263901686Z No resources found
2026-05-28T15:39:42.339792379Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-05-28T15:39:42.339792379Z See 'oc rsync -h' for help and examples
2026-05-28T15:39:42.382799650Z error: the server doesn't have a resource type "performanceprofile"
2026-05-28T15:39:42.408137920Z INFO: Worker host service log collection to complete.
2026-05-28T15:39:42.446849821Z INFO: Waiting for HAProxy config collection to complete ...
2026-05-28T15:39:42.457811685Z INFO: Found 1 replicas - prometheus-k8s-0
2026-05-28T15:39:42.468468660Z Wrote inspect data to must-gather.
2026-05-28T15:39:42.538391689Z ERROR: No running kube-apiserver pods found
2026-05-28T15:39:42.859677852Z Gathering data for ns/openshift-monitoring...
2026-05-28T15:39:42.863730135Z Wrote inspect data to must-gather.
2026-05-28T15:39:42.920368244Z tar: Removing leading `/' from member names
2026-05-28T15:39:42.924400839Z INFO: HAProxy config collection complete.
2026-05-28T15:39:42.945662680Z No resources found
2026-05-28T15:39:43.036658368Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-05-28T15:39:43.103019542Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.140787954Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.150256273Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T15:39:43.185415505Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.188976507Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-05-28T15:39:43.205945083Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.232599237Z error: the server doesn't have a resource type "machineconfigs"
2026-05-28T15:39:43.236098596Z INFO: 'previous-logs' folder not found on ip-10-0-130-203.ec2.internal, skipping...
2026-05-28T15:39:43.431233558Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.441276835Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.488827905Z INFO: 'previous-logs' folder not found on ip-10-0-130-98.ec2.internal, skipping...
2026-05-28T15:39:43.618383341Z INFO: Getting rules from prometheus-k8s-0
2026-05-28T15:39:43.657965529Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.691822244Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.734201437Z INFO: 'previous-logs' folder not found on ip-10-0-132-154.ec2.internal, skipping...
2026-05-28T15:39:43.734304687Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-05-28T15:39:43.734354282Z INFO: Machine Config Daemon termination log collection complete.
2026-05-28T15:39:43.804900016Z error: the server doesn't have a resource type "kubeletconfigs"
2026-05-28T15:39:43.921945767Z Wrote inspect data to must-gather.
2026-05-28T15:39:43.932704381Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-28T15:39:43.985269171Z INFO: Getting status/config from prometheus-k8s-0
2026-05-28T15:39:43.995028724Z Wrote inspect data to must-gather.
2026-05-28T15:39:44.198883550Z Wrote inspect data to must-gather.
2026-05-28T15:39:44.202853010Z INFO: INTERCONNECT MODE
2026-05-28T15:39:44.202853010Z INFO: Gathering ovn-kubernetes DBs
2026-05-28T15:39:44.235898832Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-05-28T15:39:44.355686837Z INFO: Gathering OVN_Northbound from ovnkube-node-gx9nw...
2026-05-28T15:39:44.355947213Z INFO: Gathering OVN_Northbound from ovnkube-node-ld542...
2026-05-28T15:39:44.356188531Z INFO: Gathering OVN_Northbound from ovnkube-node-tx2n2...
2026-05-28T15:39:44.356471369Z INFO: Gathering OVN_Southbound from ovnkube-node-gx9nw...
2026-05-28T15:39:44.356744467Z INFO: Gathering OVN_Southbound from ovnkube-node-ld542...
2026-05-28T15:39:44.357008300Z INFO: Gathering OVN_Southbound from ovnkube-node-tx2n2...
2026-05-28T15:39:44.375581041Z INFO: Getting status/flags from prometheus-k8s-0
2026-05-28T15:39:44.417495330Z Wrote inspect data to must-gather.
2026-05-28T15:39:44.664542504Z tar: Removing leading `/' from member names
2026-05-28T15:39:44.669840351Z tar: Removing leading `/' from member names
2026-05-28T15:39:44.682746799Z tar: Removing leading `/' from member names
2026-05-28T15:39:44.686972089Z tar: Removing leading `/' from member names
2026-05-28T15:39:44.704915888Z tar: Removing leading `/' from member names
2026-05-28T15:39:44.706815653Z tar: Removing leading `/' from member names
2026-05-28T15:39:44.719201360Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-05-28T15:39:44.776578143Z Wrote inspect data to must-gather.
2026-05-28T15:39:44.940255893Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-05-28T15:39:44.970040760Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-05-28T15:39:45.089580020Z Wrote inspect data to must-gather.
2026-05-28T15:39:45.141551494Z Gathering data for ns/openshift-network-console...
2026-05-28T15:39:45.172734254Z INFO: Waiting for network log collection to complete ...
2026-05-28T15:39:45.172783201Z INFO: Waiting for ovnk database copies to complete ...
2026-05-28T15:39:45.173305560Z INFO: Copying ovnk databases complete.
2026-05-28T15:39:45.174992282Z 3.7M	must-gather/network_logs/ovnk_database_store
2026-05-28T15:39:45.177789856Z ovnk_database_store/
2026-05-28T15:39:45.177839818Z ovnk_database_store/ovnkube-node-ld542_nbdb
2026-05-28T15:39:45.188968997Z ovnk_database_store/ovnkube-node-ld542_sbdb
2026-05-28T15:39:45.217817291Z ovnk_database_store/ovnkube-node-gx9nw_sbdb
2026-05-28T15:39:45.224368626Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-05-28T15:39:45.239461586Z ovnk_database_store/ovnkube-node-gx9nw_nbdb
2026-05-28T15:39:45.249345246Z ovnk_database_store/ovnkube-node-tx2n2_sbdb
2026-05-28T15:39:45.269811977Z ovnk_database_store/ovnkube-node-tx2n2_nbdb
2026-05-28T15:39:45.280008234Z INFO: Network log collection complete.
2026-05-28T15:39:45.373538588Z Wrote inspect data to must-gather.
2026-05-28T15:39:45.534735107Z Gathering data for ns/openshift-console-operator...
2026-05-28T15:39:45.584247855Z INFO: Getting status from alertmanager-main-0
2026-05-28T15:39:45.667521407Z Wrote inspect data to must-gather.
2026-05-28T15:39:45.895294340Z Gathering data for ns/openshift-console...
2026-05-28T15:39:45.906640310Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-05-28T15:39:45.908835019Z INFO: Fallback to identify the container image from release info
2026-05-28T15:39:46.262578615Z Gathering data for ns/openshift-cluster-storage-operator...
2026-05-28T15:39:46.424978796Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:934b226bb305ecfe687cdb9684d7d4cf04b130e6254c2817dd1a6112d4829bda
2026-05-28T15:39:46.546356027Z daemonset.apps/perf-node-gather-daemonset created
2026-05-28T15:39:46.581970039Z Gathering data for ns/openshift-dns-operator...
2026-05-28T15:39:46.672578016Z Waiting for performance profile collector pods to become ready: 1
2026-05-28T15:39:46.822529531Z Gathering data for ns/openshift-dns...
2026-05-28T15:39:47.360455204Z Gathering data for ns/openshift-image-registry...
2026-05-28T15:39:47.763934142Z Gathering data for ns/openshift-ingress-operator...
2026-05-28T15:39:47.794786948Z Waiting for performance profile collector pods to become ready: 2
2026-05-28T15:39:48.055079707Z Gathering data for ns/openshift-ingress...
2026-05-28T15:39:48.339333186Z Gathering data for ns/openshift-ingress-canary...
2026-05-28T15:39:48.657992605Z Gathering data for ns/openshift-insights...
2026-05-28T15:39:48.901818005Z Waiting for performance profile collector pods to become ready: 3
2026-05-28T15:39:48.937085622Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-28T15:39:50.005340235Z Waiting for performance profile collector pods to become ready: 4
2026-05-28T15:39:50.455421975Z Gathering data for ns/openshift-config...
2026-05-28T15:39:50.721863280Z Gathering data for ns/openshift-config-managed...
2026-05-28T15:39:51.099805547Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-05-28T15:39:51.127792299Z Waiting for performance profile collector pods to become ready: 5
2026-05-28T15:39:51.328810970Z Gathering data for ns/openshift-kube-apiserver...
2026-05-28T15:39:51.536519423Z Gathering data for ns/openshift-kube-controller-manager...
2026-05-28T15:39:51.774960051Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-05-28T15:39:52.010234631Z Gathering data for ns/openshift-kube-scheduler...
2026-05-28T15:39:52.239438726Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-05-28T15:39:52.249499441Z Waiting for performance profile collector pods to become ready: 6
2026-05-28T15:39:52.472616783Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-05-28T15:39:52.744644403Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-05-28T15:39:52.999461762Z Gathering data for ns/openshift-user-workload-monitoring...
2026-05-28T15:39:53.359754869Z Waiting for performance profile collector pods to become ready: 7
2026-05-28T15:39:53.504348646Z Gathering data for ns/openshift-multus...
2026-05-28T15:39:53.941361626Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-28T15:39:54.480762133Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-05-28T15:39:54.744190747Z Collecting performance related data for node ip-10-0-130-203.ec2.internal
2026-05-28T15:39:54.751103158Z Gathering data for ns/openshift-ovn-kubernetes...
2026-05-28T15:39:54.751399345Z Collecting performance related data for node ip-10-0-132-154.ec2.internal
2026-05-28T15:39:54.759203649Z Collecting performance related data for node ip-10-0-130-98.ec2.internal
2026-05-28T15:39:56.211184086Z Gathering data for ns/openshift-host-network...
2026-05-28T15:39:56.485565161Z Gathering data for ns/openshift-network-diagnostics...
2026-05-28T15:39:56.996314601Z Gathering data for ns/openshift-network-node-identity...
2026-05-28T15:39:57.417165003Z Gathering data for ns/openshift-network-operator...
2026-05-28T15:39:57.762158512Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-05-28T15:39:57.978685973Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-05-28T15:39:58.370475013Z Gathering data for ns/openshift-apiserver-operator...
2026-05-28T15:39:58.626366036Z Gathering data for ns/openshift-apiserver...
2026-05-28T15:39:58.946834064Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-05-28T15:39:58.963806871Z Gathering data for ns/openshift-controller-manager-operator...
2026-05-28T15:39:59.232992283Z Gathering data for ns/openshift-controller-manager...
2026-05-28T15:39:59.463541660Z Gathering data for ns/openshift-cluster-samples-operator...
2026-05-28T15:40:00.070995663Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-05-28T15:40:00.314680935Z Gathering data for ns/openshift-service-ca-operator...
2026-05-28T15:40:00.584226153Z Gathering data for ns/openshift-service-ca...
2026-05-28T15:40:00.896002536Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-05-28T15:40:01.705097775Z Wrote inspect data to must-gather.
2026-05-28T15:40:01.705167984Z error: inspection completed with the errors occurred while gathering data:
2026-05-28T15:40:01.705167984Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-05-28T15:40:02.438019427Z Collecting kubelet logs for node ip-10-0-130-203.ec2.internal
2026-05-28T15:40:02.439652388Z Collecting kubelet logs for node ip-10-0-132-154.ec2.internal
2026-05-28T15:40:02.441356853Z Collecting kubelet logs for node ip-10-0-130-98.ec2.internal
2026-05-28T15:40:02.738033408Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-05-28T15:40:02.743012037Z INFO: Node performance data collection complete.
2026-05-28T15:40:02.866517401Z error: the server doesn't have a resource type "clusters"
2026-05-28T15:40:03.018679737Z Caches written to disk