2026-07-13T19:22:55.112344062Z [disk usage checker] Started
2026-07-13T19:22:55.115548400Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T19:22:55.314267904Z Gathering data for ns/openshift-cluster-version...
2026-07-13T19:22:55.489175080Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-13T19:22:55.586601489Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T19:22:55.679199043Z Gathering data for ns/default...
2026-07-13T19:22:55.963465155Z Gathering data for ns/openshift...
2026-07-13T19:22:56.343949679Z Gathering data for ns/kube-system...
2026-07-13T19:22:57.008668800Z Gathering data for ns/openshift-etcd...
2026-07-13T19:22:57.283378958Z Wrote inspect data to must-gather.
2026-07-13T19:22:57.283408541Z error: inspection completed with the errors occurred while gathering data:
2026-07-13T19:22:57.283408541Z     namespaces "assisted-installer" not found
2026-07-13T19:22:57.636482674Z Waiting on subprocesses to finish execution.
2026-07-13T19:22:57.641016215Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-13T19:22:57.643999690Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-13T19:22:57.654075617Z INFO: Gathering HAProxy config files
2026-07-13T19:22:57.656884868Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-13T19:22:57.665521114Z INFO: Collecting host service logs for crio
2026-07-13T19:22:57.665886682Z INFO: Collecting host service logs for kubelet
2026-07-13T19:22:57.666247969Z INFO: Collecting host service logs for rpm-ostreed
2026-07-13T19:22:57.666620151Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-13T19:22:57.666992889Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-13T19:22:57.667359672Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-13T19:22:57.667710941Z INFO: Collecting host service logs for NetworkManager
2026-07-13T19:22:57.668119965Z INFO: Collecting host service logs for openvswitch
2026-07-13T19:22:57.668464881Z INFO: Collecting host service logs for ovs-configuration
2026-07-13T19:22:57.668796572Z INFO: Collecting host service logs for ovsdb-server
2026-07-13T19:22:57.669160326Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-13T19:22:57.669533246Z INFO: Waiting for worker host service log collection to complete ...
2026-07-13T19:22:57.669568653Z Executing Istio gather script
2026-07-13T19:22:57.802225009Z INFO: Waiting for node performance related collection to complete ...
2026-07-13T19:22:57.875164619Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-13T19:22:58.128252796Z No resources found
2026-07-13T19:22:58.191838266Z Wrote inspect data to must-gather.
2026-07-13T19:22:58.345157399Z No resources found in openshift-etcd namespace.
2026-07-13T19:22:58.399881161Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-13T19:22:58.586658266Z Wrote inspect data to must-gather.
2026-07-13T19:22:58.876877131Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-13T19:22:58.876965749Z INFO: on-disk MachineConfig config collection complete.
2026-07-13T19:22:58.881845591Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-13T19:22:58.971663204Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-drgf9
2026-07-13T19:22:58.971740601Z insights-runtime-extractor-7jkxs
2026-07-13T19:22:58.971761119Z insights-runtime-extractor-9t9z9
2026-07-13T19:22:58.971779508Z insights-runtime-extractor-h4s4b
2026-07-13T19:22:58.973074494Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-13T19:22:59.019970620Z INFO: Worker host service log collection to complete.
2026-07-13T19:22:59.061085232Z INFO: "metallb-operator" not detected. Skipping.
2026-07-13T19:22:59.072163217Z ERROR: No running kube-apiserver pods found
2026-07-13T19:22:59.127724313Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-13T19:22:59.309902210Z error: the server doesn't have a resource type "performanceprofile"
2026-07-13T19:22:59.596126750Z Wrote inspect data to must-gather.
2026-07-13T19:22:59.603772178Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-13T19:22:59.787080124Z Wrote inspect data to must-gather.
2026-07-13T19:22:59.827948956Z tar: Removing leading `/' from member names
2026-07-13T19:22:59.831520362Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-13T19:22:59.831520362Z See 'oc rsync -h' for help and examples
2026-07-13T19:22:59.837522823Z INFO: HAProxy config collection complete.
2026-07-13T19:22:59.874644483Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T19:22:59.921531695Z Gathering data for ns/openshift-monitoring...
2026-07-13T19:22:59.956339163Z No resources found
2026-07-13T19:22:59.979723761Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-13T19:22:59.991878405Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-13T19:23:00.023925384Z Wrote inspect data to must-gather.
2026-07-13T19:23:00.030233140Z Wrote inspect data to must-gather.
2026-07-13T19:23:00.072170660Z Wrote inspect data to must-gather.
2026-07-13T19:23:00.097691067Z INFO: 'previous-logs' folder not found on ip-10-0-128-140.ec2.internal, skipping...
2026-07-13T19:23:00.123571364Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T19:23:00.229774994Z error: the server doesn't have a resource type "machineconfigs"
2026-07-13T19:23:00.305547020Z Wrote inspect data to must-gather.
2026-07-13T19:23:00.342424458Z INFO: 'previous-logs' folder not found on ip-10-0-135-39.ec2.internal, skipping...
2026-07-13T19:23:00.412266411Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-13T19:23:00.414935225Z INFO: Getting rules from prometheus-k8s-0
2026-07-13T19:23:00.571720730Z Wrote inspect data to must-gather.
2026-07-13T19:23:00.627457977Z INFO: 'previous-logs' folder not found on ip-10-0-140-178.ec2.internal, skipping...
2026-07-13T19:23:00.627496359Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-13T19:23:00.627525837Z INFO: Machine Config Daemon termination log collection complete.
2026-07-13T19:23:00.642676311Z Wrote inspect data to must-gather.
2026-07-13T19:23:00.801606199Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-13T19:23:00.806782250Z Wrote inspect data to must-gather.
2026-07-13T19:23:00.831468279Z INFO: Getting status/config from prometheus-k8s-0
2026-07-13T19:23:00.915946800Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-13T19:23:01.028954680Z Wrote inspect data to must-gather.
2026-07-13T19:23:01.086448140Z INFO: INTERCONNECT MODE
2026-07-13T19:23:01.086472635Z INFO: Gathering ovn-kubernetes DBs
2026-07-13T19:23:01.092881304Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:01.247523724Z Wrote inspect data to must-gather.
2026-07-13T19:23:01.264068354Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-13T19:23:01.264203080Z INFO: Gathering OVN_Northbound from ovnkube-node-hjdw9...
2026-07-13T19:23:01.264441375Z INFO: Gathering OVN_Northbound from ovnkube-node-n4rjb...
2026-07-13T19:23:01.264710865Z INFO: Gathering OVN_Northbound from ovnkube-node-pfwh8...
2026-07-13T19:23:01.264997616Z INFO: Gathering OVN_Southbound from ovnkube-node-hjdw9...
2026-07-13T19:23:01.265240311Z INFO: Gathering OVN_Southbound from ovnkube-node-n4rjb...
2026-07-13T19:23:01.265527757Z INFO: Gathering OVN_Southbound from ovnkube-node-pfwh8...
2026-07-13T19:23:01.373632093Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:01.378030503Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-13T19:23:01.621630109Z Wrote inspect data to must-gather.
2026-07-13T19:23:01.660659947Z tar: Removing leading `/' from member names
2026-07-13T19:23:01.701373360Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-13T19:23:01.737546045Z tar: Removing leading `/' from member names
2026-07-13T19:23:01.743920885Z tar: Removing leading `/' from member names
2026-07-13T19:23:01.748521336Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:01.753998089Z tar: Removing leading `/' from member names
2026-07-13T19:23:01.765948838Z tar: Removing leading `/' from member names
2026-07-13T19:23:01.770874454Z tar: Removing leading `/' from member names
2026-07-13T19:23:01.784004033Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-13T19:23:01.844744857Z Wrote inspect data to must-gather.
2026-07-13T19:23:02.013289917Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:02.016023331Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-13T19:23:02.062544032Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-13T19:23:02.115714830Z Wrote inspect data to must-gather.
2026-07-13T19:23:02.136935609Z INFO: Waiting for network log collection to complete ...
2026-07-13T19:23:02.137001469Z INFO: Waiting for ovnk database copies to complete ...
2026-07-13T19:23:02.137500571Z INFO: Copying ovnk databases complete.
2026-07-13T19:23:02.141058591Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-13T19:23:02.144832284Z ovnk_database_store/
2026-07-13T19:23:02.144883408Z ovnk_database_store/ovnkube-node-n4rjb_nbdb
2026-07-13T19:23:02.151968161Z ovnk_database_store/ovnkube-node-pfwh8_sbdb
2026-07-13T19:23:02.167436868Z ovnk_database_store/ovnkube-node-hjdw9_sbdb
2026-07-13T19:23:02.185890032Z ovnk_database_store/ovnkube-node-pfwh8_nbdb
2026-07-13T19:23:02.193634412Z ovnk_database_store/ovnkube-node-hjdw9_nbdb
2026-07-13T19:23:02.203530502Z ovnk_database_store/ovnkube-node-n4rjb_sbdb
2026-07-13T19:23:02.223065959Z INFO: Network log collection complete.
2026-07-13T19:23:02.247122633Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:02.316929397Z W0713 19:23:02.316871    1501 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-13T19:23:02.324038371Z Wrote inspect data to must-gather.
2026-07-13T19:23:02.325078082Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-13T19:23:02.413564738Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T19:23:02.467956834Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:02.470945986Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-13T19:23:02.598884826Z Gathering data for ns/openshift-network-console...
2026-07-13T19:23:02.634919187Z Wrote inspect data to must-gather.
2026-07-13T19:23:02.709345440Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:02.710936262Z INFO: Getting status from alertmanager-main-0
2026-07-13T19:23:02.919740729Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:02.923762531Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-13T19:23:02.929968704Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-13T19:23:02.932952213Z INFO: Fallback to identify the container image from release info
2026-07-13T19:23:03.071318914Z Gathering data for ns/openshift-console-operator...
2026-07-13T19:23:03.097957329Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:03.282651552Z Wrote inspect data to /must-gather/istio.
2026-07-13T19:23:03.446175596Z Gathering data for ns/openshift-console...
2026-07-13T19:23:03.619737668Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-13T19:23:03.734452175Z daemonset.apps/perf-node-gather-daemonset created
2026-07-13T19:23:03.874545922Z Waiting for performance profile collector pods to become ready: 1
2026-07-13T19:23:03.953769834Z Done executing Istio gather script
2026-07-13T19:23:04.001524348Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-13T19:23:04.489226667Z Gathering data for ns/openshift-dns-operator...
2026-07-13T19:23:04.808032824Z Gathering data for ns/openshift-dns...
2026-07-13T19:23:04.975896804Z Waiting for performance profile collector pods to become ready: 2
2026-07-13T19:23:05.128860655Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T19:23:05.393622589Z Gathering data for ns/openshift-image-registry...
2026-07-13T19:23:05.844025394Z Gathering data for ns/openshift-ingress-operator...
2026-07-13T19:23:06.100484074Z Waiting for performance profile collector pods to become ready: 3
2026-07-13T19:23:06.212004846Z Gathering data for ns/openshift-ingress...
2026-07-13T19:23:06.531067870Z Gathering data for ns/openshift-ingress-canary...
2026-07-13T19:23:06.912323346Z Gathering data for ns/openshift-insights...
2026-07-13T19:23:07.203834498Z Waiting for performance profile collector pods to become ready: 4
2026-07-13T19:23:08.322229239Z Waiting for performance profile collector pods to become ready: 5
2026-07-13T19:23:08.790503867Z Gathering data for ns/openshift-config...
2026-07-13T19:23:09.041391073Z Gathering data for ns/openshift-config-managed...
2026-07-13T19:23:09.449650854Z Waiting for performance profile collector pods to become ready: 6
2026-07-13T19:23:09.468287249Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-13T19:23:09.759458373Z Gathering data for ns/openshift-kube-apiserver...
2026-07-13T19:23:10.015007354Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-13T19:23:10.133654617Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T19:23:10.288987530Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-13T19:23:10.571071176Z Waiting for performance profile collector pods to become ready: 7
2026-07-13T19:23:10.579884716Z Gathering data for ns/openshift-kube-scheduler...
2026-07-13T19:23:10.838699399Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-13T19:23:11.161596454Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-13T19:23:11.482268629Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-13T19:23:11.672073087Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-13T19:23:11.813061975Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-13T19:23:11.921230216Z Collecting performance related data for node ip-10-0-140-178.ec2.internal
2026-07-13T19:23:11.927891555Z Collecting performance related data for node ip-10-0-135-39.ec2.internal
2026-07-13T19:23:11.934712280Z Collecting performance related data for node ip-10-0-128-140.ec2.internal
2026-07-13T19:23:12.467862619Z Gathering data for ns/openshift-multus...
2026-07-13T19:23:14.029225915Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-13T19:23:15.138684687Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T19:23:15.378199792Z Gathering data for ns/openshift-host-network...
2026-07-13T19:23:15.679239606Z Gathering data for ns/openshift-network-diagnostics...
2026-07-13T19:23:16.223507889Z Gathering data for ns/openshift-network-node-identity...
2026-07-13T19:23:16.636379272Z Gathering data for ns/openshift-network-operator...
2026-07-13T19:23:17.027903350Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-13T19:23:17.527641667Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-13T19:23:18.006816733Z Gathering data for ns/openshift-apiserver-operator...
2026-07-13T19:23:18.289714814Z Gathering data for ns/openshift-apiserver...
2026-07-13T19:23:18.665006749Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-13T19:23:18.940240460Z Gathering data for ns/openshift-controller-manager...
2026-07-13T19:23:19.228892426Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-13T19:23:19.698297599Z Collecting kubelet logs for node ip-10-0-140-178.ec2.internal
2026-07-13T19:23:19.700073398Z Collecting kubelet logs for node ip-10-0-135-39.ec2.internal
2026-07-13T19:23:19.702208338Z Collecting kubelet logs for node ip-10-0-128-140.ec2.internal
2026-07-13T19:23:19.939086713Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-13T19:23:20.028658265Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-nk4wb namespace
2026-07-13T19:23:20.033760001Z INFO: Node performance data collection complete.
2026-07-13T19:23:20.146675937Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T19:23:20.222244794Z Gathering data for ns/openshift-service-ca-operator...
2026-07-13T19:23:20.543848491Z Gathering data for ns/openshift-service-ca...
2026-07-13T19:23:20.886231539Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-13T19:23:21.752328060Z Wrote inspect data to must-gather.
2026-07-13T19:23:21.752377000Z error: inspection completed with the errors occurred while gathering data:
2026-07-13T19:23:21.752377000Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-13T19:23:21.879264478Z error: the server doesn't have a resource type "clusters"
2026-07-13T19:23:22.007117034Z Caches written to disk