2026-06-08T16:13:01.716699480Z [disk usage checker] Started
2026-06-08T16:13:01.720548484Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T16:13:01.909404860Z Gathering data for ns/openshift-cluster-version...
2026-06-08T16:13:02.033522546Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T16:13:02.113219073Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T16:13:02.193032818Z Gathering data for ns/default...
2026-06-08T16:13:02.473558222Z Gathering data for ns/openshift...
2026-06-08T16:13:02.858752390Z Gathering data for ns/kube-system...
2026-06-08T16:13:03.515910801Z Gathering data for ns/openshift-etcd...
2026-06-08T16:13:03.789156055Z Wrote inspect data to must-gather.
2026-06-08T16:13:03.789279898Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T16:13:03.789279898Z     namespaces "assisted-installer" not found
2026-06-08T16:13:04.258212051Z Waiting on subprocesses to finish execution.
2026-06-08T16:13:04.265386196Z INFO: Collecting host service logs for crio
2026-06-08T16:13:04.265708298Z INFO: Collecting host service logs for kubelet
2026-06-08T16:13:04.266050343Z INFO: Collecting host service logs for rpm-ostreed
2026-06-08T16:13:04.266409883Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-08T16:13:04.266752267Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-08T16:13:04.267075768Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-08T16:13:04.267526163Z INFO: Collecting host service logs for NetworkManager
2026-06-08T16:13:04.267867678Z INFO: Collecting host service logs for openvswitch
2026-06-08T16:13:04.268220926Z INFO: Collecting host service logs for ovs-configuration
2026-06-08T16:13:04.268624421Z INFO: Collecting host service logs for ovsdb-server
2026-06-08T16:13:04.269051673Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-08T16:13:04.269448159Z INFO: Waiting for worker host service log collection to complete ...
2026-06-08T16:13:04.282330296Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-08T16:13:04.289165295Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-08T16:13:04.291931960Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-08T16:13:04.301893475Z Executing Istio gather script
2026-06-08T16:13:04.316508189Z INFO: Gathering HAProxy config files
2026-06-08T16:13:04.453267375Z INFO: Waiting for node performance related collection to complete ...
2026-06-08T16:13:04.757443821Z INFO: "metallb-operator" not detected. Skipping.
2026-06-08T16:13:04.957617211Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-08T16:13:04.974867113Z No resources found in openshift-etcd namespace.
2026-06-08T16:13:05.117524465Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-08T16:13:05.327958886Z No resources found
2026-06-08T16:13:05.427898117Z Wrote inspect data to must-gather.
2026-06-08T16:13:05.443267834Z INFO: Collecting Insights Archives from insights-operator-896549976-mtcnl
2026-06-08T16:13:05.443310037Z insights-runtime-extractor-2h6k8
2026-06-08T16:13:05.443332125Z insights-runtime-extractor-hxp68
2026-06-08T16:13:05.443352365Z insights-runtime-extractor-mmhfd
2026-06-08T16:13:05.573779904Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-08T16:13:05.573842466Z INFO: on-disk MachineConfig config collection complete.
2026-06-08T16:13:05.575493273Z error: the server doesn't have a resource type "performanceprofile"
2026-06-08T16:13:05.598356989Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-08T16:13:05.643603879Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-08T16:13:05.655393072Z ERROR: No running kube-apiserver pods found
2026-06-08T16:13:05.675971247Z INFO: Worker host service log collection to complete.
2026-06-08T16:13:06.016926511Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-08T16:13:06.120455884Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-08T16:13:06.120455884Z See 'oc rsync -h' for help and examples
2026-06-08T16:13:06.132883581Z Wrote inspect data to must-gather.
2026-06-08T16:13:06.246057005Z Gathering data for ns/openshift-monitoring...
2026-06-08T16:13:06.262431746Z Wrote inspect data to must-gather.
2026-06-08T16:13:06.371776922Z No resources found
2026-06-08T16:13:06.389160298Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-08T16:13:06.560011545Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T16:13:06.569002056Z INFO: 'previous-logs' folder not found on ip-10-0-130-204.ec2.internal, skipping...
2026-06-08T16:13:06.692499129Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-08T16:13:06.726078282Z Wrote inspect data to must-gather.
2026-06-08T16:13:06.737971476Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T16:13:06.769170431Z error: the server doesn't have a resource type "machineconfigs"
2026-06-08T16:13:06.893180534Z Wrote inspect data to must-gather.
2026-06-08T16:13:06.904977872Z Wrote inspect data to must-gather.
2026-06-08T16:13:06.909492867Z Wrote inspect data to must-gather.
2026-06-08T16:13:06.992291444Z tar: Removing leading `/' from member names
2026-06-08T16:13:06.996696283Z INFO: HAProxy config collection complete.
2026-06-08T16:13:07.011927103Z INFO: Getting rules from prometheus-k8s-0
2026-06-08T16:13:07.073487221Z INFO: 'previous-logs' folder not found on ip-10-0-137-208.ec2.internal, skipping...
2026-06-08T16:13:07.075418107Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-08T16:13:07.228357650Z Wrote inspect data to must-gather.
2026-06-08T16:13:07.343747879Z Wrote inspect data to must-gather.
2026-06-08T16:13:07.354300281Z INFO: 'previous-logs' folder not found on ip-10-0-138-187.ec2.internal, skipping...
2026-06-08T16:13:07.354490553Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-08T16:13:07.354551939Z INFO: Machine Config Daemon termination log collection complete.
2026-06-08T16:13:07.374609280Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-08T16:13:07.453232402Z INFO: Getting status/config from prometheus-k8s-0
2026-06-08T16:13:07.493456079Z Wrote inspect data to must-gather.
2026-06-08T16:13:07.512297827Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-08T16:13:07.726010252Z Wrote inspect data to must-gather.
2026-06-08T16:13:07.746187522Z Wrote inspect data to must-gather.
2026-06-08T16:13:07.859952534Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-08T16:13:07.939537588Z Wrote inspect data to must-gather.
2026-06-08T16:13:08.027078602Z INFO: INTERCONNECT MODE
2026-06-08T16:13:08.027078602Z INFO: Gathering ovn-kubernetes DBs
2026-06-08T16:13:08.035076430Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-08T16:13:08.112062824Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-08T16:13:08.127607042Z Wrote inspect data to must-gather.
2026-06-08T16:13:08.178298059Z INFO: Gathering OVN_Northbound from ovnkube-node-6sdmv...
2026-06-08T16:13:08.178547376Z INFO: Gathering OVN_Northbound from ovnkube-node-d5d4c...
2026-06-08T16:13:08.178797674Z INFO: Gathering OVN_Northbound from ovnkube-node-dqj2d...
2026-06-08T16:13:08.179169121Z INFO: Gathering OVN_Southbound from ovnkube-node-6sdmv...
2026-06-08T16:13:08.182479298Z INFO: Gathering OVN_Southbound from ovnkube-node-d5d4c...
2026-06-08T16:13:08.182791476Z INFO: Gathering OVN_Southbound from ovnkube-node-dqj2d...
2026-06-08T16:13:08.237767341Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:08.456300374Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-08T16:13:08.491514117Z Wrote inspect data to must-gather.
2026-06-08T16:13:08.495082587Z tar: Removing leading `/' from member names
2026-06-08T16:13:08.526061356Z tar: Removing leading `/' from member names
2026-06-08T16:13:08.559550083Z tar: Removing leading `/' from member names
2026-06-08T16:13:08.587436688Z tar: Removing leading `/' from member names
2026-06-08T16:13:08.590905829Z tar: Removing leading `/' from member names
2026-06-08T16:13:08.632813072Z tar: Removing leading `/' from member names
2026-06-08T16:13:08.663261263Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:08.666479860Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-08T16:13:08.669608207Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T16:13:08.756343648Z Wrote inspect data to must-gather.
2026-06-08T16:13:08.809533156Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-08T16:13:08.909264239Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:09.017430691Z Wrote inspect data to must-gather.
2026-06-08T16:13:09.057389609Z INFO: Waiting for network log collection to complete ...
2026-06-08T16:13:09.057441510Z INFO: Waiting for ovnk database copies to complete ...
2026-06-08T16:13:09.057928919Z INFO: Copying ovnk databases complete.
2026-06-08T16:13:09.059810967Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-06-08T16:13:09.062172377Z ovnk_database_store/
2026-06-08T16:13:09.062172377Z ovnk_database_store/ovnkube-node-6sdmv_sbdb
2026-06-08T16:13:09.081546824Z ovnk_database_store/ovnkube-node-d5d4c_sbdb
2026-06-08T16:13:09.099318136Z ovnk_database_store/ovnkube-node-dqj2d_sbdb
2026-06-08T16:13:09.118318836Z ovnk_database_store/ovnkube-node-6sdmv_nbdb
2026-06-08T16:13:09.126571180Z ovnk_database_store/ovnkube-node-dqj2d_nbdb
2026-06-08T16:13:09.137374742Z ovnk_database_store/ovnkube-node-d5d4c_nbdb
2026-06-08T16:13:09.143682698Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:09.145869902Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-08T16:13:09.149430074Z INFO: Network log collection complete.
2026-06-08T16:13:09.233398350Z W0608 16:13:09.233344    1512 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-08T16:13:09.239454385Z Wrote inspect data to must-gather.
2026-06-08T16:13:09.241343437Z INFO: Getting status from alertmanager-main-0
2026-06-08T16:13:09.340569776Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T16:13:09.362317755Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:09.513599662Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-08T16:13:09.516378678Z INFO: Fallback to identify the container image from release info
2026-06-08T16:13:09.528230729Z Gathering data for ns/openshift-network-console...
2026-06-08T16:13:09.570448393Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:09.574447248Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-08T16:13:09.793437810Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:09.950244281Z Gathering data for ns/openshift-console-operator...
2026-06-08T16:13:09.975606833Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:09.977734779Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-08T16:13:10.141311782Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:10.210656200Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-08T16:13:10.343844663Z Wrote inspect data to /must-gather/istio.
2026-06-08T16:13:10.345498390Z daemonset.apps/perf-node-gather-daemonset created
2026-06-08T16:13:10.352122615Z Gathering data for ns/openshift-console...
2026-06-08T16:13:10.487378294Z Waiting for performance profile collector pods to become ready: 1
2026-06-08T16:13:10.928634586Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-08T16:13:11.051451731Z Done executing Istio gather script
2026-06-08T16:13:11.399938689Z Gathering data for ns/openshift-dns-operator...
2026-06-08T16:13:11.607854291Z Waiting for performance profile collector pods to become ready: 2
2026-06-08T16:13:11.672628813Z Gathering data for ns/openshift-dns...
2026-06-08T16:13:11.753298623Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T16:13:12.264464078Z Gathering data for ns/openshift-image-registry...
2026-06-08T16:13:12.732546960Z Gathering data for ns/openshift-ingress-operator...
2026-06-08T16:13:12.735091608Z Waiting for performance profile collector pods to become ready: 3
2026-06-08T16:13:13.087054704Z Gathering data for ns/openshift-ingress...
2026-06-08T16:13:13.412374312Z Gathering data for ns/openshift-ingress-canary...
2026-06-08T16:13:13.788140767Z Gathering data for ns/openshift-insights...
2026-06-08T16:13:13.853480085Z Waiting for performance profile collector pods to become ready: 4
2026-06-08T16:13:14.959910358Z Waiting for performance profile collector pods to become ready: 5
2026-06-08T16:13:15.601331754Z Gathering data for ns/openshift-jobset-operator...
2026-06-08T16:13:16.067678555Z Waiting for performance profile collector pods to become ready: 6
2026-06-08T16:13:16.381503998Z Gathering data for ns/openshift-config...
2026-06-08T16:13:16.629332065Z Gathering data for ns/openshift-config-managed...
2026-06-08T16:13:16.757727689Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T16:13:17.058585054Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-08T16:13:17.189425354Z Waiting for performance profile collector pods to become ready: 7
2026-06-08T16:13:17.310288199Z Gathering data for ns/openshift-kube-apiserver...
2026-06-08T16:13:17.553424400Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-08T16:13:17.829336477Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-08T16:13:18.140563847Z Gathering data for ns/openshift-kube-scheduler...
2026-06-08T16:13:18.300015917Z Waiting for performance profile collector pods to become ready: 8
2026-06-08T16:13:18.388185166Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-08T16:13:18.664951425Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-08T16:13:18.969371650Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-08T16:13:19.260863667Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-08T16:13:19.418596534Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-08T16:13:19.668522508Z Collecting performance related data for node ip-10-0-138-187.ec2.internal
2026-06-08T16:13:19.678507901Z Collecting performance related data for node ip-10-0-137-208.ec2.internal
2026-06-08T16:13:19.685617662Z Collecting performance related data for node ip-10-0-130-204.ec2.internal
2026-06-08T16:13:19.790407800Z Gathering data for ns/openshift-multus...
2026-06-08T16:13:21.372140163Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-08T16:13:21.762298762Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T16:13:22.709042918Z Gathering data for ns/openshift-host-network...
2026-06-08T16:13:22.998604727Z Gathering data for ns/openshift-network-diagnostics...
2026-06-08T16:13:23.515405917Z Gathering data for ns/openshift-network-node-identity...
2026-06-08T16:13:23.936744378Z Gathering data for ns/openshift-network-operator...
2026-06-08T16:13:24.317951429Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-08T16:13:24.582928073Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-08T16:13:25.059294855Z Gathering data for ns/openshift-apiserver-operator...
2026-06-08T16:13:25.352335685Z Gathering data for ns/openshift-apiserver...
2026-06-08T16:13:25.747665730Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-08T16:13:25.991519777Z Gathering data for ns/openshift-controller-manager...
2026-06-08T16:13:26.247612898Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-08T16:13:26.768444938Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-06-08T16:13:26.911617098Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-08T16:13:27.183582100Z Gathering data for ns/openshift-service-ca-operator...
2026-06-08T16:13:27.487080110Z Gathering data for ns/openshift-service-ca...
2026-06-08T16:13:27.649921387Z Collecting kubelet logs for node ip-10-0-138-187.ec2.internal
2026-06-08T16:13:27.652317749Z Collecting kubelet logs for node ip-10-0-137-208.ec2.internal
2026-06-08T16:13:27.654439539Z Collecting kubelet logs for node ip-10-0-130-204.ec2.internal
2026-06-08T16:13:27.890512008Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-08T16:13:27.976490640Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4wnkl namespace
2026-06-08T16:13:27.980945394Z INFO: Node performance data collection complete.
2026-06-08T16:13:28.800827027Z Wrote inspect data to must-gather.
2026-06-08T16:13:28.800874313Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T16:13:28.800874313Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-08T16:13:28.924887121Z error: the server doesn't have a resource type "clusters"
2026-06-08T16:13:29.053080774Z Caches written to disk