2026-06-11T16:34:28.303727476Z [disk usage checker] Started
2026-06-11T16:34:28.307535164Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-11T16:34:28.516953813Z Gathering data for ns/openshift-cluster-version...
2026-06-11T16:34:28.672009222Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-11T16:34:28.773144747Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T16:34:28.854526126Z Gathering data for ns/default...
2026-06-11T16:34:29.173640960Z Gathering data for ns/openshift...
2026-06-11T16:34:29.657892038Z Gathering data for ns/kube-system...
2026-06-11T16:34:30.416795546Z Gathering data for ns/openshift-etcd...
2026-06-11T16:34:30.748414827Z Wrote inspect data to must-gather.
2026-06-11T16:34:30.748595061Z error: inspection completed with the errors occurred while gathering data:
2026-06-11T16:34:30.748595061Z     namespaces "assisted-installer" not found
2026-06-11T16:34:30.968085110Z Waiting on subprocesses to finish execution.
2026-06-11T16:34:30.978775290Z INFO: Gathering HAProxy config files
2026-06-11T16:34:30.979615076Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-11T16:34:30.986185732Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-11T16:34:30.987529977Z INFO: Collecting host service logs for crio
2026-06-11T16:34:30.987791113Z INFO: Collecting host service logs for kubelet
2026-06-11T16:34:30.988107238Z INFO: Collecting host service logs for rpm-ostreed
2026-06-11T16:34:30.988555449Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-11T16:34:30.988882113Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-11T16:34:30.989199935Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-11T16:34:30.989525695Z INFO: Collecting host service logs for NetworkManager
2026-06-11T16:34:30.989833117Z INFO: Collecting host service logs for openvswitch
2026-06-11T16:34:30.990149697Z INFO: Collecting host service logs for ovs-configuration
2026-06-11T16:34:30.990481476Z INFO: Collecting host service logs for ovsdb-server
2026-06-11T16:34:30.990499989Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-11T16:34:30.990942757Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-11T16:34:30.991320114Z INFO: Waiting for worker host service log collection to complete ...
2026-06-11T16:34:31.001990787Z Executing Istio gather script
2026-06-11T16:34:31.146005127Z INFO: Waiting for node performance related collection to complete ...
2026-06-11T16:34:31.502678811Z INFO: Collecting Insights Archives from insights-operator-896549976-gznn2
2026-06-11T16:34:31.502737292Z insights-runtime-extractor-7jwhb
2026-06-11T16:34:31.502757785Z insights-runtime-extractor-99sj9
2026-06-11T16:34:31.502778031Z insights-runtime-extractor-ddb55
2026-06-11T16:34:31.729358258Z INFO: "metallb-operator" not detected. Skipping.
2026-06-11T16:34:31.816628607Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-11T16:34:31.967786959Z error: the server doesn't have a resource type "performanceprofile"
2026-06-11T16:34:32.030171394Z Wrote inspect data to must-gather.
2026-06-11T16:34:32.091048911Z No resources found in openshift-etcd namespace.
2026-06-11T16:34:32.282945468Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-11T16:34:32.282945468Z See 'oc rsync -h' for help and examples
2026-06-11T16:34:32.369391837Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-11T16:34:32.369528180Z INFO: on-disk MachineConfig config collection complete.
2026-06-11T16:34:32.410885568Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-11T16:34:32.420796981Z No resources found
2026-06-11T16:34:32.433128273Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-11T16:34:32.533424355Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-11T16:34:32.730300148Z INFO: Worker host service log collection to complete.
2026-06-11T16:34:32.817609669Z Wrote inspect data to must-gather.
2026-06-11T16:34:32.819660520Z ERROR: No running kube-apiserver pods found
2026-06-11T16:34:32.974228186Z No resources found
2026-06-11T16:34:33.072718146Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-11T16:34:33.074798498Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-11T16:34:33.104280645Z Wrote inspect data to must-gather.
2026-06-11T16:34:33.322334154Z Wrote inspect data to must-gather.
2026-06-11T16:34:33.334564297Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T16:34:33.336218273Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-11T16:34:33.346432099Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-11T16:34:33.370107605Z Wrote inspect data to must-gather.
2026-06-11T16:34:33.387449609Z Wrote inspect data to must-gather.
2026-06-11T16:34:33.465921107Z Wrote inspect data to must-gather.
2026-06-11T16:34:33.486722859Z INFO: 'previous-logs' folder not found on ip-10-0-131-210.ec2.internal, skipping...
2026-06-11T16:34:33.491920120Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-11T16:34:33.664156510Z tar: Removing leading `/' from member names
2026-06-11T16:34:33.665336429Z error: the server doesn't have a resource type "machineconfigs"
2026-06-11T16:34:33.669707486Z INFO: HAProxy config collection complete.
2026-06-11T16:34:33.724894616Z Wrote inspect data to must-gather.
2026-06-11T16:34:33.776574587Z INFO: 'previous-logs' folder not found on ip-10-0-136-90.ec2.internal, skipping...
2026-06-11T16:34:33.853423206Z Gathering data for ns/openshift-monitoring...
2026-06-11T16:34:33.855499954Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-11T16:34:33.899641281Z INFO: Getting rules from prometheus-k8s-0
2026-06-11T16:34:34.019123150Z Wrote inspect data to must-gather.
2026-06-11T16:34:34.066557125Z INFO: 'previous-logs' folder not found on ip-10-0-136-95.ec2.internal, skipping...
2026-06-11T16:34:34.066668058Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-11T16:34:34.066735054Z INFO: Machine Config Daemon termination log collection complete.
2026-06-11T16:34:34.091837817Z Wrote inspect data to must-gather.
2026-06-11T16:34:34.252868543Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-11T16:34:34.265473830Z Wrote inspect data to must-gather.
2026-06-11T16:34:34.383059550Z INFO: Getting status/config from prometheus-k8s-0
2026-06-11T16:34:34.476963934Z Wrote inspect data to must-gather.
2026-06-11T16:34:34.563534966Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-11T16:34:34.565726238Z INFO: INTERCONNECT MODE
2026-06-11T16:34:34.565750701Z INFO: Gathering ovn-kubernetes DBs
2026-06-11T16:34:34.685493107Z Wrote inspect data to must-gather.
2026-06-11T16:34:34.724357153Z INFO: Gathering OVN_Northbound from ovnkube-node-djqnm...
2026-06-11T16:34:34.724656973Z INFO: Gathering OVN_Northbound from ovnkube-node-fzdnk...
2026-06-11T16:34:34.724901558Z INFO: Gathering OVN_Northbound from ovnkube-node-zd55g...
2026-06-11T16:34:34.725180272Z INFO: Gathering OVN_Southbound from ovnkube-node-djqnm...
2026-06-11T16:34:34.725456314Z INFO: Gathering OVN_Southbound from ovnkube-node-fzdnk...
2026-06-11T16:34:34.725697056Z INFO: Gathering OVN_Southbound from ovnkube-node-zd55g...
2026-06-11T16:34:34.771280243Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:34.845395328Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-11T16:34:35.042113785Z Wrote inspect data to must-gather.
2026-06-11T16:34:35.068110540Z tar: Removing leading `/' from member names
2026-06-11T16:34:35.088820236Z tar: Removing leading `/' from member names
2026-06-11T16:34:35.100750022Z tar: Removing leading `/' from member names
2026-06-11T16:34:35.117763168Z tar: Removing leading `/' from member names
2026-06-11T16:34:35.124110478Z tar: Removing leading `/' from member names
2026-06-11T16:34:35.125987557Z tar: Removing leading `/' from member names
2026-06-11T16:34:35.172953263Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:35.175868884Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-11T16:34:35.190568051Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-11T16:34:35.293634297Z Wrote inspect data to must-gather.
2026-06-11T16:34:35.390971972Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:35.445596777Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-11T16:34:35.507951280Z Wrote inspect data to must-gather.
2026-06-11T16:34:35.544655403Z INFO: Waiting for network log collection to complete ...
2026-06-11T16:34:35.544686959Z INFO: Waiting for ovnk database copies to complete ...
2026-06-11T16:34:35.545109800Z INFO: Copying ovnk databases complete.
2026-06-11T16:34:35.547487104Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-06-11T16:34:35.550841653Z ovnk_database_store/
2026-06-11T16:34:35.550889301Z ovnk_database_store/ovnkube-node-djqnm_sbdb
2026-06-11T16:34:35.578304983Z ovnk_database_store/ovnkube-node-fzdnk_sbdb
2026-06-11T16:34:35.595996633Z ovnk_database_store/ovnkube-node-djqnm_nbdb
2026-06-11T16:34:35.603709318Z ovnk_database_store/ovnkube-node-fzdnk_nbdb
2026-06-11T16:34:35.612682321Z ovnk_database_store/ovnkube-node-zd55g_sbdb
2026-06-11T16:34:35.628397093Z ovnk_database_store/ovnkube-node-zd55g_nbdb
2026-06-11T16:34:35.631033665Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:35.634055371Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-11T16:34:35.639478286Z INFO: Network log collection complete.
2026-06-11T16:34:35.714771118Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-11T16:34:35.730824318Z Wrote inspect data to must-gather.
2026-06-11T16:34:35.735998050Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-11T16:34:35.822007405Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:35.950053220Z Wrote inspect data to must-gather.
2026-06-11T16:34:36.058342996Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:36.063808998Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-11T16:34:36.123724742Z INFO: Getting status from alertmanager-main-0
2026-06-11T16:34:36.240175263Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-11T16:34:36.242307879Z INFO: Fallback to identify the container image from release info
2026-06-11T16:34:36.307077194Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:36.369698762Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-11T16:34:36.505426487Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:36.507690704Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-11T16:34:36.548903814Z Gathering data for ns/openshift-network-console...
2026-06-11T16:34:36.675850473Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:36.867855170Z Wrote inspect data to /must-gather/istio.
2026-06-11T16:34:36.885930396Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-11T16:34:37.026967270Z daemonset.apps/perf-node-gather-daemonset created
2026-06-11T16:34:37.044118727Z Gathering data for ns/openshift-console-operator...
2026-06-11T16:34:37.155967721Z Waiting for performance profile collector pods to become ready: 1
2026-06-11T16:34:37.518388209Z Gathering data for ns/openshift-console...
2026-06-11T16:34:37.581662108Z Done executing Istio gather script
2026-06-11T16:34:38.077593777Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-11T16:34:38.261702703Z Waiting for performance profile collector pods to become ready: 2
2026-06-11T16:34:38.356115137Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-11T16:34:38.701583654Z Gathering data for ns/openshift-dns-operator...
2026-06-11T16:34:39.076301525Z Gathering data for ns/openshift-dns...
2026-06-11T16:34:39.368916348Z Waiting for performance profile collector pods to become ready: 3
2026-06-11T16:34:39.827173966Z Gathering data for ns/openshift-image-registry...
2026-06-11T16:34:40.407168522Z Gathering data for ns/openshift-ingress-operator...
2026-06-11T16:34:40.496405819Z Waiting for performance profile collector pods to become ready: 4
2026-06-11T16:34:40.768566082Z Gathering data for ns/openshift-ingress...
2026-06-11T16:34:41.116924030Z Gathering data for ns/openshift-ingress-canary...
2026-06-11T16:34:41.527504460Z Gathering data for ns/openshift-insights...
2026-06-11T16:34:41.608504226Z Waiting for performance profile collector pods to become ready: 5
2026-06-11T16:34:42.722357897Z Waiting for performance profile collector pods to become ready: 6
2026-06-11T16:34:43.361135059Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-11T16:34:43.473939602Z Gathering data for ns/openshift-config...
2026-06-11T16:34:43.771025016Z Gathering data for ns/openshift-config-managed...
2026-06-11T16:34:43.861472919Z Waiting for performance profile collector pods to become ready: 7
2026-06-11T16:34:44.291088509Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-11T16:34:44.566792671Z Gathering data for ns/openshift-kube-apiserver...
2026-06-11T16:34:44.884582496Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-11T16:34:44.985755461Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-11T16:34:45.205403925Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-11T16:34:45.250926586Z Collecting performance related data for node ip-10-0-136-90.ec2.internal
2026-06-11T16:34:45.260546114Z Collecting performance related data for node ip-10-0-136-95.ec2.internal
2026-06-11T16:34:45.270136608Z Collecting performance related data for node ip-10-0-131-210.ec2.internal
2026-06-11T16:34:45.554363764Z Gathering data for ns/openshift-kube-scheduler...
2026-06-11T16:34:45.944981966Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-11T16:34:46.331700363Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-11T16:34:46.735501104Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-11T16:34:47.155719063Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-11T16:34:47.768658242Z Gathering data for ns/openshift-multus...
2026-06-11T16:34:48.365989960Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-11T16:34:49.631502478Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-11T16:34:51.033112489Z Gathering data for ns/openshift-host-network...
2026-06-11T16:34:51.357109236Z Gathering data for ns/openshift-network-diagnostics...
2026-06-11T16:34:51.924777182Z Gathering data for ns/openshift-network-node-identity...
2026-06-11T16:34:52.387731442Z Gathering data for ns/openshift-network-operator...
2026-06-11T16:34:52.785629087Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-11T16:34:53.054246716Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-11T16:34:53.371297919Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-11T16:34:53.578810977Z Gathering data for ns/openshift-apiserver-operator...
2026-06-11T16:34:53.597573468Z Collecting kubelet logs for node ip-10-0-136-90.ec2.internal
2026-06-11T16:34:53.600048130Z Collecting kubelet logs for node ip-10-0-136-95.ec2.internal
2026-06-11T16:34:53.602733118Z Collecting kubelet logs for node ip-10-0-131-210.ec2.internal
2026-06-11T16:34:53.948285682Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-z5p7g namespace
2026-06-11T16:34:53.954013912Z INFO: Node performance data collection complete.
2026-06-11T16:34:54.020157667Z Gathering data for ns/openshift-apiserver...
2026-06-11T16:34:54.469187331Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-11T16:34:54.750335385Z Gathering data for ns/openshift-controller-manager...
2026-06-11T16:34:55.032099893Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-11T16:34:55.697241938Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-11T16:34:56.015692175Z Gathering data for ns/openshift-service-ca-operator...
2026-06-11T16:34:56.458347413Z Gathering data for ns/openshift-service-ca...
2026-06-11T16:34:56.849696357Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-11T16:34:57.898438338Z Wrote inspect data to must-gather.
2026-06-11T16:34:57.898482186Z error: inspection completed with the errors occurred while gathering data:
2026-06-11T16:34:57.898482186Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-11T16:34:58.023582731Z error: the server doesn't have a resource type "clusters"
2026-06-11T16:34:58.154815725Z Caches written to disk