2026-04-24T22:16:25.071004009Z [disk usage checker] Started
2026-04-24T22:16:25.074155777Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-24T22:16:25.300501680Z Gathering data for ns/openshift-cluster-version...
2026-04-24T22:16:25.496142179Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:16:25.638172413Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:16:25.740659788Z Gathering data for ns/default...
2026-04-24T22:16:26.128719825Z Gathering data for ns/openshift...
2026-04-24T22:16:26.643737990Z Gathering data for ns/kube-system...
2026-04-24T22:16:27.374240003Z Gathering data for ns/openshift-etcd...
2026-04-24T22:16:27.643693362Z Waiting on subprocesses to finish execution.
2026-04-24T22:16:27.650279394Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-24T22:16:27.651851373Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-24T22:16:27.657780857Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-24T22:16:27.678819358Z INFO: Collecting host service logs for crio
2026-04-24T22:16:27.679141200Z INFO: Collecting host service logs for kubelet
2026-04-24T22:16:27.679449197Z INFO: Collecting host service logs for rpm-ostreed
2026-04-24T22:16:27.679782296Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-24T22:16:27.680067648Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-24T22:16:27.680373543Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-24T22:16:27.680686266Z INFO: Collecting host service logs for NetworkManager
2026-04-24T22:16:27.681031705Z INFO: Collecting host service logs for openvswitch
2026-04-24T22:16:27.681341028Z INFO: Collecting host service logs for ovs-configuration
2026-04-24T22:16:27.681659029Z INFO: Collecting host service logs for ovsdb-server
2026-04-24T22:16:27.681997006Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-24T22:16:27.682351835Z INFO: Waiting for worker host service log collection to complete ...
2026-04-24T22:16:27.682808143Z INFO: Waiting for node performance related collection to complete ...
2026-04-24T22:16:27.691185975Z INFO: Gathering HAProxy config files
2026-04-24T22:16:28.325459430Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-24T22:16:28.325542198Z INFO: on-disk MachineConfig config collection complete.
2026-04-24T22:16:28.437053368Z Wrote inspect data to must-gather.
2026-04-24T22:16:28.437151828Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:16:28.437151828Z     namespaces "assisted-installer" not found
2026-04-24T22:16:28.442112759Z No resources found in openshift-etcd namespace.
2026-04-24T22:16:28.453687936Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-w6q7n
2026-04-24T22:16:28.453742632Z insights-runtime-extractor-6ck65
2026-04-24T22:16:28.453765147Z insights-runtime-extractor-sxbgk
2026-04-24T22:16:28.453785129Z insights-runtime-extractor-vqxb9
2026-04-24T22:16:28.536209976Z Wrote inspect data to must-gather.
2026-04-24T22:16:28.618826606Z INFO: "metallb-operator" not detected. Skipping.
2026-04-24T22:16:28.657627256Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-24T22:16:28.855342386Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-24T22:16:28.932721757Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-24T22:16:28.932721757Z See 'oc rsync -h' for help and examples
2026-04-24T22:16:28.995902141Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-24T22:16:29.041114334Z error: the server doesn't have a resource type "performanceprofile"
2026-04-24T22:16:29.056394355Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-24T22:16:29.086110607Z No resources found
2026-04-24T22:16:29.131757644Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-24T22:16:29.227025122Z INFO: Worker host service log collection to complete.
2026-04-24T22:16:29.359995414Z ERROR: No running kube-apiserver pods found
2026-04-24T22:16:29.697575423Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-24T22:16:29.985296271Z tar: Removing leading `/' from member names
2026-04-24T22:16:29.986510032Z Wrote inspect data to must-gather.
2026-04-24T22:16:29.991549266Z INFO: HAProxy config collection complete.
2026-04-24T22:16:30.108534542Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:16:30.162211754Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-24T22:16:30.267165932Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-24T22:16:30.275154496Z No resources found
2026-04-24T22:16:30.300167435Z Wrote inspect data to must-gather.
2026-04-24T22:16:30.348633169Z Wrote inspect data to must-gather.
2026-04-24T22:16:30.468429474Z Wrote inspect data to must-gather.
2026-04-24T22:16:30.486895106Z INFO: 'previous-logs' folder not found on ip-10-0-133-36.ec2.internal, skipping...
2026-04-24T22:16:30.643375050Z Wrote inspect data to must-gather.
2026-04-24T22:16:30.685492379Z error: the server doesn't have a resource type "machineconfigs"
2026-04-24T22:16:30.829796679Z Gathering data for ns/kuadrant-system...
2026-04-24T22:16:30.853076404Z INFO: Getting rules from prometheus-k8s-0
2026-04-24T22:16:30.883456602Z INFO: 'previous-logs' folder not found on ip-10-0-134-232.ec2.internal, skipping...
2026-04-24T22:16:31.036646608Z Wrote inspect data to must-gather.
2026-04-24T22:16:31.064311422Z Wrote inspect data to must-gather.
2026-04-24T22:16:31.313129668Z INFO: 'previous-logs' folder not found on ip-10-0-142-242.ec2.internal, skipping...
2026-04-24T22:16:31.313239214Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-24T22:16:31.313295953Z INFO: Machine Config Daemon termination log collection complete.
2026-04-24T22:16:31.409575879Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-24T22:16:31.426555519Z Wrote inspect data to must-gather.
2026-04-24T22:16:31.431084993Z Wrote inspect data to must-gather.
2026-04-24T22:16:31.490845793Z INFO: Getting status/config from prometheus-k8s-0
2026-04-24T22:16:31.670642656Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-24T22:16:31.746024822Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-24T22:16:31.830441015Z Wrote inspect data to must-gather.
2026-04-24T22:16:31.998015668Z Gathering data for ns/openshift-monitoring...
2026-04-24T22:16:32.037856679Z Wrote inspect data to must-gather.
2026-04-24T22:16:32.077476369Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-24T22:16:32.246577269Z INFO: INTERCONNECT MODE
2026-04-24T22:16:32.246613333Z INFO: Gathering ovn-kubernetes DBs
2026-04-24T22:16:32.347271701Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-24T22:16:32.384738310Z Wrote inspect data to must-gather.
2026-04-24T22:16:32.447009868Z INFO: Gathering OVN_Northbound from ovnkube-node-bt4zz...
2026-04-24T22:16:32.447269399Z INFO: Gathering OVN_Northbound from ovnkube-node-c6xp2...
2026-04-24T22:16:32.447522291Z INFO: Gathering OVN_Northbound from ovnkube-node-mxnmf...
2026-04-24T22:16:32.447854208Z INFO: Gathering OVN_Southbound from ovnkube-node-bt4zz...
2026-04-24T22:16:32.448143077Z INFO: Gathering OVN_Southbound from ovnkube-node-c6xp2...
2026-04-24T22:16:32.448419903Z INFO: Gathering OVN_Southbound from ovnkube-node-mxnmf...
2026-04-24T22:16:32.723151543Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-24T22:16:32.810048742Z tar: Removing leading `/' from member names
2026-04-24T22:16:32.822148159Z tar: Removing leading `/' from member names
2026-04-24T22:16:32.841335406Z tar: Removing leading `/' from member names
2026-04-24T22:16:32.850313542Z tar: Removing leading `/' from member names
2026-04-24T22:16:32.893112324Z Wrote inspect data to must-gather.
2026-04-24T22:16:32.910544994Z tar: Removing leading `/' from member names
2026-04-24T22:16:32.912108195Z tar: Removing leading `/' from member names
2026-04-24T22:16:33.125984192Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-24T22:16:33.250591413Z Wrote inspect data to must-gather.
2026-04-24T22:16:33.581837680Z INFO: Waiting for network log collection to complete ...
2026-04-24T22:16:33.581889377Z INFO: Waiting for ovnk database copies to complete ...
2026-04-24T22:16:33.582380569Z INFO: Copying ovnk databases complete.
2026-04-24T22:16:33.584447863Z 5.0M	must-gather/network_logs/ovnk_database_store
2026-04-24T22:16:33.587440931Z ovnk_database_store/
2026-04-24T22:16:33.587486113Z ovnk_database_store/ovnkube-node-mxnmf_nbdb
2026-04-24T22:16:33.601247603Z ovnk_database_store/ovnkube-node-c6xp2_sbdb
2026-04-24T22:16:33.613929824Z INFO: Getting status from alertmanager-main-0
2026-04-24T22:16:33.618755833Z Wrote inspect data to must-gather.
2026-04-24T22:16:33.627348867Z ovnk_database_store/ovnkube-node-c6xp2_nbdb
2026-04-24T22:16:33.638450928Z ovnk_database_store/ovnkube-node-bt4zz_nbdb
2026-04-24T22:16:33.649840392Z ovnk_database_store/ovnkube-node-bt4zz_sbdb
2026-04-24T22:16:33.682473080Z ovnk_database_store/ovnkube-node-mxnmf_sbdb
2026-04-24T22:16:33.716099280Z INFO: Network log collection complete.
2026-04-24T22:16:33.921047857Z Wrote inspect data to must-gather.
2026-04-24T22:16:34.177979735Z W0424 22:16:34.177916    1433 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-24T22:16:34.196404862Z Wrote inspect data to must-gather.
2026-04-24T22:16:34.480996644Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-24T22:16:34.484506047Z INFO: Fallback to identify the container image from release info
2026-04-24T22:16:34.501897572Z Gathering data for ns/openshift-network-console...
2026-04-24T22:16:35.028738788Z Gathering data for ns/openshift-console-operator...
2026-04-24T22:16:35.098763472Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-24T22:16:35.200157563Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-24T22:16:35.245744717Z daemonset.apps/perf-node-gather-daemonset created
2026-04-24T22:16:35.334750879Z Wrote inspect data to must-gather.
2026-04-24T22:16:35.388591854Z Waiting for performance profile collector pods to become ready: 1
2026-04-24T22:16:35.505855502Z Gathering data for ns/openshift-console...
2026-04-24T22:16:36.079376904Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-24T22:16:36.474014629Z Gathering data for ns/openshift-dns-operator...
2026-04-24T22:16:36.533242319Z Waiting for performance profile collector pods to become ready: 2
2026-04-24T22:16:36.818569499Z Gathering data for ns/openshift-dns...
2026-04-24T22:16:37.466934194Z Gathering data for ns/openshift-image-registry...
2026-04-24T22:16:37.673951289Z Waiting for performance profile collector pods to become ready: 3
2026-04-24T22:16:37.984650906Z Gathering data for ns/openshift-ingress-operator...
2026-04-24T22:16:38.347794803Z Gathering data for ns/openshift-ingress...
2026-04-24T22:16:38.810002146Z Waiting for performance profile collector pods to become ready: 4
2026-04-24T22:16:38.933882306Z Gathering data for ns/openshift-ingress-canary...
2026-04-24T22:16:39.396497922Z Gathering data for ns/openshift-insights...
2026-04-24T22:16:39.943999954Z Waiting for performance profile collector pods to become ready: 5
2026-04-24T22:16:40.204571657Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-24T22:16:41.069309434Z Waiting for performance profile collector pods to become ready: 6
2026-04-24T22:16:42.187240205Z Gathering data for ns/openshift-lws-operator...
2026-04-24T22:16:42.201166410Z Waiting for performance profile collector pods to become ready: 7
2026-04-24T22:16:42.765867724Z Gathering data for ns/kserve...
2026-04-24T22:16:43.325401046Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-24T22:16:43.606955947Z Collecting performance related data for node ip-10-0-134-232.ec2.internal
2026-04-24T22:16:43.613725130Z Collecting performance related data for node ip-10-0-133-36.ec2.internal
2026-04-24T22:16:43.620752121Z Collecting performance related data for node ip-10-0-142-242.ec2.internal
2026-04-24T22:16:44.861553805Z Gathering data for ns/openshift-config...
2026-04-24T22:16:45.209494349Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-24T22:16:45.288131795Z Gathering data for ns/openshift-config-managed...
2026-04-24T22:16:45.785007778Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-24T22:16:46.163945779Z Gathering data for ns/openshift-kube-apiserver...
2026-04-24T22:16:46.552571322Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-24T22:16:46.930850250Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-24T22:16:47.323261304Z Gathering data for ns/openshift-kube-scheduler...
2026-04-24T22:16:47.678960106Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-24T22:16:48.063335629Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-24T22:16:48.449037288Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-24T22:16:48.862028719Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-24T22:16:49.468021842Z Gathering data for ns/openshift-multus...
2026-04-24T22:16:50.214929923Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-24T22:16:51.023281245Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-24T22:16:52.049576218Z Collecting kubelet logs for node ip-10-0-134-232.ec2.internal
2026-04-24T22:16:52.052360364Z Collecting kubelet logs for node ip-10-0-133-36.ec2.internal
2026-04-24T22:16:52.054146841Z Collecting kubelet logs for node ip-10-0-142-242.ec2.internal
2026-04-24T22:16:52.455329280Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-24T22:16:52.460638732Z INFO: Node performance data collection complete.
2026-04-24T22:16:52.758724782Z Gathering data for ns/openshift-host-network...
2026-04-24T22:16:53.133908398Z Gathering data for ns/openshift-network-diagnostics...
2026-04-24T22:16:53.709141661Z Gathering data for ns/openshift-network-node-identity...
2026-04-24T22:16:54.188892186Z Gathering data for ns/openshift-network-operator...
2026-04-24T22:16:54.615177761Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-24T22:16:54.927263674Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-24T22:16:55.219957015Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-04-24T22:16:55.487243048Z Gathering data for ns/openshift-apiserver-operator...
2026-04-24T22:16:55.828819730Z Gathering data for ns/openshift-apiserver...
2026-04-24T22:16:56.303229758Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-24T22:16:56.641368770Z Gathering data for ns/openshift-controller-manager...
2026-04-24T22:16:56.955009824Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-24T22:16:57.640055542Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-24T22:16:58.002029101Z Gathering data for ns/openshift-service-ca-operator...
2026-04-24T22:16:58.351818181Z Gathering data for ns/openshift-service-ca...
2026-04-24T22:16:58.805599664Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-24T22:16:59.831899413Z Wrote inspect data to must-gather.
2026-04-24T22:16:59.831943622Z error: inspection completed with the errors occurred while gathering data:
2026-04-24T22:16:59.831943622Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-24T22:16:59.974042708Z error: the server doesn't have a resource type "clusters"
2026-04-24T22:17:00.143469199Z Caches written to disk