2026-04-22T19:04:51.238038438Z [disk usage checker] Started
2026-04-22T19:04:51.272529894Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:04:51.439079285Z Gathering data for ns/openshift-cluster-version...
2026-04-22T19:04:51.565005281Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T19:04:51.653584222Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:04:51.732750168Z Gathering data for ns/default...
2026-04-22T19:04:52.030447814Z Gathering data for ns/openshift...
2026-04-22T19:04:52.447969325Z Gathering data for ns/kube-system...
2026-04-22T19:04:53.169790492Z Gathering data for ns/openshift-etcd...
2026-04-22T19:04:53.475325345Z Wrote inspect data to must-gather.
2026-04-22T19:04:53.475359305Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T19:04:53.475359305Z     namespaces "assisted-installer" not found
2026-04-22T19:04:53.657353751Z Waiting on subprocesses to finish execution.
2026-04-22T19:04:53.664074387Z INFO: Gathering HAProxy config files
2026-04-22T19:04:53.665724997Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-04-22T19:04:53.666326140Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-04-22T19:04:53.667484622Z INFO: Gathering machine config daemon's old logs from all nodes
2026-04-22T19:04:53.720628439Z INFO: Collecting host service logs for crio
2026-04-22T19:04:53.721239132Z INFO: Collecting host service logs for kubelet
2026-04-22T19:04:53.721630707Z INFO: Collecting host service logs for rpm-ostreed
2026-04-22T19:04:53.721984598Z INFO: Collecting host service logs for ostree-finalize-staged
2026-04-22T19:04:53.722324007Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-04-22T19:04:53.722689676Z INFO: Collecting host service logs for machine-config-daemon-host
2026-04-22T19:04:53.723024609Z INFO: Collecting host service logs for NetworkManager
2026-04-22T19:04:53.723349193Z INFO: Collecting host service logs for openvswitch
2026-04-22T19:04:53.723705808Z INFO: Collecting host service logs for ovs-configuration
2026-04-22T19:04:53.724041723Z INFO: Collecting host service logs for ovsdb-server
2026-04-22T19:04:53.724363941Z INFO: Collecting host service logs for ovs-vswitchd
2026-04-22T19:04:53.724780363Z INFO: Waiting for worker host service log collection to complete ...
2026-04-22T19:04:54.196358529Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-04-22T19:04:54.196397664Z INFO: on-disk MachineConfig config collection complete.
2026-04-22T19:04:54.206014938Z INFO: Waiting for node performance related collection to complete ...
2026-04-22T19:04:54.272846172Z INFO: "metallb-operator" not detected. Skipping.
2026-04-22T19:04:54.301426768Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-04-22T19:04:54.528522746Z No resources found in openshift-etcd namespace.
2026-04-22T19:04:54.740358255Z INFO: Collecting Insights Archives from insights-operator-585dfdc468-g6jpq
2026-04-22T19:04:54.740358255Z insights-runtime-extractor-6qmlv
2026-04-22T19:04:54.740358255Z insights-runtime-extractor-97npl
2026-04-22T19:04:54.740358255Z insights-runtime-extractor-9fq46
2026-04-22T19:04:54.789448085Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-04-22T19:04:54.807189041Z No resources found
2026-04-22T19:04:54.828849041Z error: the server doesn't have a resource type "performanceprofile"
2026-04-22T19:04:55.033976561Z INFO: "sriov-network-operator" not detected. Skipping.
2026-04-22T19:04:55.045869947Z Wrote inspect data to must-gather.
2026-04-22T19:04:55.074821302Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-04-22T19:04:55.074821302Z See 'oc rsync -h' for help and examples
2026-04-22T19:04:55.087191771Z INFO: Worker host service log collection to complete.
2026-04-22T19:04:55.143661996Z INFO: Waiting for HAProxy config collection to complete ...
2026-04-22T19:04:55.150460339Z INFO: Found 1 replicas - prometheus-k8s-0
2026-04-22T19:04:55.292805971Z ERROR: No running kube-apiserver pods found
2026-04-22T19:04:55.447009645Z Wrote inspect data to must-gather.
2026-04-22T19:04:55.673851154Z Gathering data for ns/openshift-monitoring...
2026-04-22T19:04:55.778967931Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:04:55.826662780Z INFO: 'previous-logs' folder not found on ip-10-0-130-112.ec2.internal, skipping...
2026-04-22T19:04:55.857445090Z No resources found
2026-04-22T19:04:55.872475199Z Wrote inspect data to must-gather.
2026-04-22T19:04:55.915714001Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-04-22T19:04:55.921783087Z tar: Removing leading `/' from member names
2026-04-22T19:04:55.933413036Z INFO: HAProxy config collection complete.
2026-04-22T19:04:55.939173096Z Wrote inspect data to must-gather.
2026-04-22T19:04:55.943209608Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-04-22T19:04:55.984294098Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.054159073Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.070870356Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.107629671Z error: the server doesn't have a resource type "machineconfigs"
2026-04-22T19:04:56.206336479Z INFO: 'previous-logs' folder not found on ip-10-0-135-21.ec2.internal, skipping...
2026-04-22T19:04:56.237852012Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.278400011Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:04:56.344387111Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.371582510Z INFO: Getting rules from prometheus-k8s-0
2026-04-22T19:04:56.477846160Z INFO: 'previous-logs' folder not found on ip-10-0-143-11.ec2.internal, skipping...
2026-04-22T19:04:56.477885438Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-04-22T19:04:56.477911281Z INFO: Machine Config Daemon termination log collection complete.
2026-04-22T19:04:56.503980687Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.586054309Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.744767248Z error: the server doesn't have a resource type "kubeletconfigs"
2026-04-22T19:04:56.747970716Z Wrote inspect data to must-gather.
2026-04-22T19:04:56.773924168Z INFO: Getting status/config from prometheus-k8s-0
2026-04-22T19:04:56.955147673Z Wrote inspect data to must-gather.
2026-04-22T19:04:57.028927654Z INFO: INTERCONNECT MODE
2026-04-22T19:04:57.028962106Z INFO: Gathering ovn-kubernetes DBs
2026-04-22T19:04:57.183697893Z Wrote inspect data to must-gather.
2026-04-22T19:04:57.233065314Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-04-22T19:04:57.254401278Z INFO: Getting status/flags from prometheus-k8s-0
2026-04-22T19:04:57.263578383Z INFO: Gathering OVN_Northbound from ovnkube-node-2hklb...
2026-04-22T19:04:57.263825737Z INFO: Gathering OVN_Northbound from ovnkube-node-6jx4b...
2026-04-22T19:04:57.264054611Z INFO: Gathering OVN_Northbound from ovnkube-node-8zxzp...
2026-04-22T19:04:57.264354096Z INFO: Gathering OVN_Southbound from ovnkube-node-2hklb...
2026-04-22T19:04:57.264631067Z INFO: Gathering OVN_Southbound from ovnkube-node-6jx4b...
2026-04-22T19:04:57.264835898Z INFO: Gathering OVN_Southbound from ovnkube-node-8zxzp...
2026-04-22T19:04:57.473749420Z Wrote inspect data to must-gather.
2026-04-22T19:04:57.574509124Z tar: Removing leading `/' from member names
2026-04-22T19:04:57.588972513Z tar: Removing leading `/' from member names
2026-04-22T19:04:57.600167636Z tar: Removing leading `/' from member names
2026-04-22T19:04:57.605428911Z tar: Removing leading `/' from member names
2026-04-22T19:04:57.607896977Z tar: Removing leading `/' from member names
2026-04-22T19:04:57.608200200Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-04-22T19:04:57.715014932Z tar: Removing leading `/' from member names
2026-04-22T19:04:57.750675774Z Wrote inspect data to must-gather.
2026-04-22T19:04:57.861174853Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-04-22T19:04:57.983271565Z W0422 19:04:57.983223    1290 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-04-22T19:04:57.988776112Z Wrote inspect data to must-gather.
2026-04-22T19:04:58.010891464Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-04-22T19:04:58.083526039Z INFO: Waiting for network log collection to complete ...
2026-04-22T19:04:58.083591421Z INFO: Waiting for ovnk database copies to complete ...
2026-04-22T19:04:58.084057679Z INFO: Copying ovnk databases complete.
2026-04-22T19:04:58.085858759Z 3.2M	must-gather/network_logs/ovnk_database_store
2026-04-22T19:04:58.089030951Z ovnk_database_store/
2026-04-22T19:04:58.089080014Z ovnk_database_store/ovnkube-node-8zxzp_sbdb
2026-04-22T19:04:58.111324706Z ovnk_database_store/ovnkube-node-2hklb_nbdb
2026-04-22T19:04:58.118946873Z ovnk_database_store/ovnkube-node-6jx4b_nbdb
2026-04-22T19:04:58.120702824Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-04-22T19:04:58.127822779Z ovnk_database_store/ovnkube-node-8zxzp_nbdb
2026-04-22T19:04:58.137673726Z ovnk_database_store/ovnkube-node-6jx4b_sbdb
2026-04-22T19:04:58.158167221Z ovnk_database_store/ovnkube-node-2hklb_sbdb
2026-04-22T19:04:58.179721540Z INFO: Network log collection complete.
2026-04-22T19:04:58.194198911Z Gathering data for ns/openshift-network-console...
2026-04-22T19:04:58.323238474Z Wrote inspect data to must-gather.
2026-04-22T19:04:58.528228130Z INFO: Getting status from alertmanager-main-0
2026-04-22T19:04:58.641486306Z Wrote inspect data to must-gather.
2026-04-22T19:04:58.648589658Z Gathering data for ns/openshift-console-operator...
2026-04-22T19:04:58.895992763Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-04-22T19:04:58.900716285Z INFO: Fallback to identify the container image from release info
2026-04-22T19:04:59.047793811Z Gathering data for ns/openshift-console...
2026-04-22T19:04:59.508187469Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8f2123db9953f98da0e43c266e6a8a070bf221533b995f87b7e358cc7498ca6d
2026-04-22T19:04:59.516063385Z Gathering data for ns/openshift-cluster-storage-operator...
2026-04-22T19:04:59.648032399Z daemonset.apps/perf-node-gather-daemonset created
2026-04-22T19:04:59.765614537Z Waiting for performance profile collector pods to become ready: 1
2026-04-22T19:04:59.981622044Z Gathering data for ns/openshift-dns-operator...
2026-04-22T19:05:00.290461262Z Gathering data for ns/openshift-dns...
2026-04-22T19:05:00.898761058Z Waiting for performance profile collector pods to become ready: 2
2026-04-22T19:05:00.973944755Z Gathering data for ns/openshift-image-registry...
2026-04-22T19:05:01.283270747Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:05:01.520212210Z Gathering data for ns/openshift-ingress-operator...
2026-04-22T19:05:01.887103631Z Gathering data for ns/openshift-ingress...
2026-04-22T19:05:02.027260268Z Waiting for performance profile collector pods to become ready: 3
2026-04-22T19:05:02.205753032Z Gathering data for ns/openshift-ingress-canary...
2026-04-22T19:05:02.578717492Z Gathering data for ns/openshift-insights...
2026-04-22T19:05:03.163287230Z Waiting for performance profile collector pods to become ready: 4
2026-04-22T19:05:04.282452984Z Waiting for performance profile collector pods to become ready: 5
2026-04-22T19:05:04.734257166Z Gathering data for ns/kserve...
2026-04-22T19:05:05.415194582Z Waiting for performance profile collector pods to become ready: 6
2026-04-22T19:05:06.287586835Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:05:06.525469479Z Gathering data for ns/openshift-config...
2026-04-22T19:05:06.533255910Z Waiting for performance profile collector pods to become ready: 7
2026-04-22T19:05:06.779646428Z Gathering data for ns/openshift-config-managed...
2026-04-22T19:05:07.181118301Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-04-22T19:05:07.470907403Z Gathering data for ns/openshift-kube-apiserver...
2026-04-22T19:05:07.661545491Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-04-22T19:05:07.733048398Z Gathering data for ns/openshift-kube-controller-manager...
2026-04-22T19:05:07.937506564Z Collecting performance related data for node ip-10-0-143-11.ec2.internal
2026-04-22T19:05:07.944275472Z Collecting performance related data for node ip-10-0-130-112.ec2.internal
2026-04-22T19:05:07.951195642Z Collecting performance related data for node ip-10-0-135-21.ec2.internal
2026-04-22T19:05:08.113946969Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-04-22T19:05:08.446325764Z Gathering data for ns/openshift-kube-scheduler...
2026-04-22T19:05:08.760824503Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-04-22T19:05:09.183833209Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-04-22T19:05:09.624760684Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-04-22T19:05:09.934043150Z Gathering data for ns/openshift-user-workload-monitoring...
2026-04-22T19:05:10.447987611Z Gathering data for ns/openshift-multus...
2026-04-22T19:05:11.293268808Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:05:11.899778887Z Gathering data for ns/openshift-ovn-kubernetes...
2026-04-22T19:05:13.514882739Z Gathering data for ns/openshift-host-network...
2026-04-22T19:05:13.886072721Z Gathering data for ns/openshift-network-diagnostics...
2026-04-22T19:05:14.452057299Z Gathering data for ns/openshift-network-node-identity...
2026-04-22T19:05:14.868550089Z Gathering data for ns/openshift-network-operator...
2026-04-22T19:05:15.243431967Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-04-22T19:05:15.559530393Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-04-22T19:05:15.976648476Z Collecting kubelet logs for node ip-10-0-143-11.ec2.internal
2026-04-22T19:05:15.979166903Z Collecting kubelet logs for node ip-10-0-130-112.ec2.internal
2026-04-22T19:05:15.981044161Z Collecting kubelet logs for node ip-10-0-135-21.ec2.internal
2026-04-22T19:05:16.024862131Z Gathering data for ns/openshift-apiserver-operator...
2026-04-22T19:05:16.301061445Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70
2026-04-22T19:05:16.350483494Z daemonset.apps "perf-node-gather-daemonset" deleted
2026-04-22T19:05:16.355335642Z INFO: Node performance data collection complete.
2026-04-22T19:05:16.377649947Z Gathering data for ns/openshift-apiserver...
2026-04-22T19:05:16.759827561Z Gathering data for ns/openshift-controller-manager-operator...
2026-04-22T19:05:17.050168749Z Gathering data for ns/openshift-controller-manager...
2026-04-22T19:05:17.312310731Z Gathering data for ns/openshift-cluster-samples-operator...
2026-04-22T19:05:17.977633241Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-04-22T19:05:18.240028165Z Gathering data for ns/openshift-service-ca-operator...
2026-04-22T19:05:18.529385934Z Gathering data for ns/openshift-service-ca...
2026-04-22T19:05:18.874806790Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-04-22T19:05:19.726227260Z Wrote inspect data to must-gather.
2026-04-22T19:05:19.726270521Z error: inspection completed with the errors occurred while gathering data:
2026-04-22T19:05:19.726270521Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-04-22T19:05:19.854053294Z error: the server doesn't have a resource type "clusters"
2026-04-22T19:05:20.006609145Z Caches written to disk