2026-06-09T21:01:47.755621123Z [disk usage checker] Started
2026-06-09T21:01:47.759307827Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T21:01:47.933647272Z Gathering data for ns/openshift-cluster-version...
2026-06-09T21:01:48.064768687Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T21:01:48.135253722Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T21:01:48.208233438Z Gathering data for ns/default...
2026-06-09T21:01:48.493541046Z Gathering data for ns/openshift...
2026-06-09T21:01:48.892545989Z Gathering data for ns/kube-system...
2026-06-09T21:01:49.639570102Z Gathering data for ns/openshift-etcd...
2026-06-09T21:01:49.898822647Z Wrote inspect data to must-gather.
2026-06-09T21:01:49.898855324Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T21:01:49.898855324Z     namespaces "assisted-installer" not found
2026-06-09T21:01:50.188903377Z Waiting on subprocesses to finish execution.
2026-06-09T21:01:50.196868749Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-09T21:01:50.198723126Z INFO: Gathering HAProxy config files
2026-06-09T21:01:50.199593574Z INFO: Collecting host service logs for crio
2026-06-09T21:01:50.199921982Z INFO: Collecting host service logs for kubelet
2026-06-09T21:01:50.200373064Z INFO: Collecting host service logs for rpm-ostreed
2026-06-09T21:01:50.200713520Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-09T21:01:50.201057757Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-09T21:01:50.201410790Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-09T21:01:50.201515847Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-09T21:01:50.201764114Z INFO: Collecting host service logs for NetworkManager
2026-06-09T21:01:50.202086470Z INFO: Collecting host service logs for openvswitch
2026-06-09T21:01:50.202425739Z INFO: Collecting host service logs for ovs-configuration
2026-06-09T21:01:50.202732123Z INFO: Collecting host service logs for ovsdb-server
2026-06-09T21:01:50.203049969Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-09T21:01:50.203444969Z INFO: Waiting for worker host service log collection to complete ...
2026-06-09T21:01:50.204921971Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-09T21:01:50.223878192Z Executing Istio gather script
2026-06-09T21:01:50.224047424Z INFO: Waiting for node performance related collection to complete ...
2026-06-09T21:01:50.604895592Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-09T21:01:50.703631505Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-09T21:01:50.974713812Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-09T21:01:51.122430584Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-09T21:01:51.149415230Z INFO: Collecting Insights Archives from insights-operator-896549976-lfxnr
2026-06-09T21:01:51.149463371Z insights-runtime-extractor-6dt2n
2026-06-09T21:01:51.149485183Z insights-runtime-extractor-6vgw9
2026-06-09T21:01:51.149505558Z insights-runtime-extractor-trd6h
2026-06-09T21:01:51.245945470Z No resources found
2026-06-09T21:01:51.268264172Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-09T21:01:51.271308991Z INFO: on-disk MachineConfig config collection complete.
2026-06-09T21:01:51.280453152Z Wrote inspect data to must-gather.
2026-06-09T21:01:51.440234280Z No resources found in openshift-etcd namespace.
2026-06-09T21:01:51.502800404Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-09T21:01:51.563807457Z INFO: "metallb-operator" not detected. Skipping.
2026-06-09T21:01:51.590677675Z INFO: Worker host service log collection to complete.
2026-06-09T21:01:51.645554707Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-09T21:01:51.645554707Z See 'oc rsync -h' for help and examples
2026-06-09T21:01:51.669706733Z error: the server doesn't have a resource type "performanceprofile"
2026-06-09T21:01:51.857626208Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-09T21:01:51.995596261Z ERROR: No running kube-apiserver pods found
2026-06-09T21:01:52.083279541Z Gathering data for ns/openshift-monitoring...
2026-06-09T21:01:52.116475286Z Wrote inspect data to must-gather.
2026-06-09T21:01:52.453815692Z No resources found
2026-06-09T21:01:52.528699240Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T21:01:52.532982135Z Wrote inspect data to must-gather.
2026-06-09T21:01:52.632235911Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-09T21:01:52.660691789Z Wrote inspect data to must-gather.
2026-06-09T21:01:52.775130087Z Wrote inspect data to must-gather.
2026-06-09T21:01:52.778836000Z INFO: Getting rules from prometheus-k8s-0
2026-06-09T21:01:52.791683024Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T21:01:52.804535666Z Wrote inspect data to must-gather.
2026-06-09T21:01:52.847924847Z INFO: 'previous-logs' folder not found on ip-10-0-128-117.ec2.internal, skipping...
2026-06-09T21:01:52.884521212Z Wrote inspect data to must-gather.
2026-06-09T21:01:52.914705425Z tar: Removing leading `/' from member names
2026-06-09T21:01:52.920025561Z INFO: HAProxy config collection complete.
2026-06-09T21:01:52.933446987Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-09T21:01:52.954668876Z error: the server doesn't have a resource type "machineconfigs"
2026-06-09T21:01:53.112391530Z Wrote inspect data to must-gather.
2026-06-09T21:01:53.129027383Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-09T21:01:53.146922279Z INFO: 'previous-logs' folder not found on ip-10-0-130-77.ec2.internal, skipping...
2026-06-09T21:01:53.265241527Z INFO: Getting status/config from prometheus-k8s-0
2026-06-09T21:01:53.369528396Z Wrote inspect data to must-gather.
2026-06-09T21:01:53.371095786Z Wrote inspect data to must-gather.
2026-06-09T21:01:53.420532053Z INFO: 'previous-logs' folder not found on ip-10-0-137-98.ec2.internal, skipping...
2026-06-09T21:01:53.420655968Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-09T21:01:53.420717828Z INFO: Machine Config Daemon termination log collection complete.
2026-06-09T21:01:53.557799054Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-09T21:01:53.626685524Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-09T21:01:53.632484306Z Wrote inspect data to must-gather.
2026-06-09T21:01:53.707525583Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-09T21:01:53.779726702Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-09T21:01:53.790660449Z Wrote inspect data to must-gather.
2026-06-09T21:01:53.947432194Z INFO: INTERCONNECT MODE
2026-06-09T21:01:53.947457471Z INFO: Gathering ovn-kubernetes DBs
2026-06-09T21:01:53.952575232Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-09T21:01:53.971266885Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:54.030357282Z Wrote inspect data to must-gather.
2026-06-09T21:01:54.122947388Z INFO: Gathering OVN_Northbound from ovnkube-node-5l764...
2026-06-09T21:01:54.123150512Z INFO: Gathering OVN_Northbound from ovnkube-node-7r7gc...
2026-06-09T21:01:54.123387471Z INFO: Gathering OVN_Northbound from ovnkube-node-lrg6f...
2026-06-09T21:01:54.123684377Z INFO: Gathering OVN_Southbound from ovnkube-node-5l764...
2026-06-09T21:01:54.123986344Z INFO: Gathering OVN_Southbound from ovnkube-node-7r7gc...
2026-06-09T21:01:54.124263729Z INFO: Gathering OVN_Southbound from ovnkube-node-lrg6f...
2026-06-09T21:01:54.253698644Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-09T21:01:54.292478966Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:54.296847404Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-09T21:01:54.312494099Z W0609 21:01:54.312455    1330 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-09T21:01:54.320104338Z Wrote inspect data to must-gather.
2026-06-09T21:01:54.381274737Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-09T21:01:54.465349100Z tar: Removing leading `/' from member names
2026-06-09T21:01:54.468417590Z tar: Removing leading `/' from member names
2026-06-09T21:01:54.477585035Z tar: Removing leading `/' from member names
2026-06-09T21:01:54.506104159Z tar: Removing leading `/' from member names
2026-06-09T21:01:54.533164077Z tar: Removing leading `/' from member names
2026-06-09T21:01:54.561575332Z tar: Removing leading `/' from member names
2026-06-09T21:01:54.601158598Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:54.619175123Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-09T21:01:54.626044498Z Gathering data for ns/openshift-network-console...
2026-06-09T21:01:54.715055681Z Wrote inspect data to must-gather.
2026-06-09T21:01:54.840932854Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:54.844004356Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-09T21:01:54.974748315Z INFO: Waiting for network log collection to complete ...
2026-06-09T21:01:54.974868652Z INFO: Waiting for ovnk database copies to complete ...
2026-06-09T21:01:54.975389146Z INFO: Copying ovnk databases complete.
2026-06-09T21:01:54.977500953Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-06-09T21:01:54.981694245Z ovnk_database_store/
2026-06-09T21:01:54.982273773Z ovnk_database_store/ovnkube-node-5l764_sbdb
2026-06-09T21:01:55.007865641Z ovnk_database_store/ovnkube-node-5l764_nbdb
2026-06-09T21:01:55.012774543Z ovnk_database_store/ovnkube-node-7r7gc_nbdb
2026-06-09T21:01:55.020874034Z ovnk_database_store/ovnkube-node-lrg6f_nbdb
2026-06-09T21:01:55.028693530Z ovnk_database_store/ovnkube-node-7r7gc_sbdb
2026-06-09T21:01:55.046698658Z ovnk_database_store/ovnkube-node-lrg6f_sbdb
2026-06-09T21:01:55.065382329Z INFO: Getting status from alertmanager-main-0
2026-06-09T21:01:55.068501905Z INFO: Network log collection complete.
2026-06-09T21:01:55.069332388Z Wrote inspect data to must-gather.
2026-06-09T21:01:55.091802506Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:55.146263032Z Gathering data for ns/openshift-console-operator...
2026-06-09T21:01:55.312746217Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:55.315133985Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-09T21:01:55.364926018Z Wrote inspect data to must-gather.
2026-06-09T21:01:55.535438045Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:55.549247667Z Gathering data for ns/openshift-console...
2026-06-09T21:01:55.644253789Z Wrote inspect data to must-gather.
2026-06-09T21:01:55.719840238Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:55.722092839Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-09T21:01:55.885033920Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:55.893816357Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-09T21:01:55.895820177Z INFO: Fallback to identify the container image from release info
2026-06-09T21:01:56.070996309Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-09T21:01:56.084687872Z Wrote inspect data to /must-gather/istio.
2026-06-09T21:01:56.520960799Z Gathering data for ns/openshift-dns-operator...
2026-06-09T21:01:56.635781593Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-09T21:01:56.743970911Z Done executing Istio gather script
2026-06-09T21:01:56.765044958Z daemonset.apps/perf-node-gather-daemonset created
2026-06-09T21:01:56.802258321Z Gathering data for ns/openshift-dns...
2026-06-09T21:01:56.880359090Z Waiting for performance profile collector pods to become ready: 1
2026-06-09T21:01:57.359238745Z Gathering data for ns/openshift-image-registry...
2026-06-09T21:01:57.750515497Z Gathering data for ns/openshift-ingress-operator...
2026-06-09T21:01:57.799702906Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T21:01:58.001456105Z Waiting for performance profile collector pods to become ready: 2
2026-06-09T21:01:58.034051488Z Gathering data for ns/openshift-ingress...
2026-06-09T21:01:58.318056579Z Gathering data for ns/openshift-ingress-canary...
2026-06-09T21:01:58.631541208Z Gathering data for ns/openshift-insights...
2026-06-09T21:01:59.117158381Z Waiting for performance profile collector pods to become ready: 3
2026-06-09T21:02:00.222009006Z Waiting for performance profile collector pods to become ready: 4
2026-06-09T21:02:00.350322497Z Gathering data for ns/openshift-config...
2026-06-09T21:02:00.554983210Z Gathering data for ns/openshift-config-managed...
2026-06-09T21:02:00.911789969Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-09T21:02:01.117336105Z Gathering data for ns/openshift-kube-apiserver...
2026-06-09T21:02:01.343088101Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-09T21:02:01.344742894Z Waiting for performance profile collector pods to become ready: 5
2026-06-09T21:02:01.566802374Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-09T21:02:01.796015102Z Gathering data for ns/openshift-kube-scheduler...
2026-06-09T21:02:01.995738050Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-09T21:02:02.214526322Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-09T21:02:02.459378593Z Waiting for performance profile collector pods to become ready: 6
2026-06-09T21:02:02.493878254Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-09T21:02:02.777588941Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-09T21:02:02.805360681Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T21:02:03.173309903Z Gathering data for ns/openshift-multus...
2026-06-09T21:02:03.555752581Z Waiting for performance profile collector pods to become ready: 7
2026-06-09T21:02:04.343748701Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-09T21:02:04.651222581Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-09T21:02:04.861869254Z Collecting performance related data for node ip-10-0-137-98.ec2.internal
2026-06-09T21:02:04.869007649Z Collecting performance related data for node ip-10-0-128-117.ec2.internal
2026-06-09T21:02:04.877563493Z Collecting performance related data for node ip-10-0-130-77.ec2.internal
2026-06-09T21:02:05.553977329Z Gathering data for ns/openshift-host-network...
2026-06-09T21:02:05.853267410Z Gathering data for ns/openshift-network-diagnostics...
2026-06-09T21:02:06.388597366Z Gathering data for ns/openshift-network-node-identity...
2026-06-09T21:02:06.707447270Z Gathering data for ns/openshift-network-operator...
2026-06-09T21:02:07.050787823Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-09T21:02:07.294936101Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-09T21:02:07.764322892Z Gathering data for ns/openshift-apiserver-operator...
2026-06-09T21:02:07.811461198Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T21:02:08.028544740Z Gathering data for ns/openshift-apiserver...
2026-06-09T21:02:08.334061423Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-09T21:02:08.546188806Z Gathering data for ns/openshift-controller-manager...
2026-06-09T21:02:08.771598924Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-09T21:02:09.365104439Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-09T21:02:09.603042571Z Gathering data for ns/openshift-service-ca-operator...
2026-06-09T21:02:09.872642442Z Gathering data for ns/openshift-service-ca...
2026-06-09T21:02:10.186634425Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-09T21:02:10.993695419Z Wrote inspect data to must-gather.
2026-06-09T21:02:10.993739003Z error: inspection completed with the errors occurred while gathering data:
2026-06-09T21:02:10.993739003Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-09T21:02:12.512051450Z Collecting kubelet logs for node ip-10-0-137-98.ec2.internal
2026-06-09T21:02:12.513720727Z Collecting kubelet logs for node ip-10-0-128-117.ec2.internal
2026-06-09T21:02:12.515437129Z Collecting kubelet logs for node ip-10-0-130-77.ec2.internal
2026-06-09T21:02:12.796267310Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-xnfg8 namespace
2026-06-09T21:02:12.800257971Z INFO: Node performance data collection complete.
2026-06-09T21:02:12.819256216Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-09T21:02:12.915283081Z error: the server doesn't have a resource type "clusters"
2026-06-09T21:02:13.041827374Z Caches written to disk