2026-07-17T15:36:57.647662537Z [disk usage checker] Started
2026-07-17T15:36:57.651579684Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-17T15:36:57.832072060Z Gathering data for ns/openshift-cluster-version...
2026-07-17T15:36:57.950685396Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-17T15:36:58.039696875Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-17T15:36:58.123126346Z Gathering data for ns/default...
2026-07-17T15:36:58.418116821Z Gathering data for ns/openshift...
2026-07-17T15:36:58.831803030Z Gathering data for ns/kube-system...
2026-07-17T15:36:59.481245058Z Gathering data for ns/openshift-etcd...
2026-07-17T15:36:59.754917449Z Wrote inspect data to must-gather.
2026-07-17T15:36:59.754948129Z error: inspection completed with the errors occurred while gathering data:
2026-07-17T15:36:59.754948129Z     namespaces "assisted-installer" not found
2026-07-17T15:37:00.122891509Z Waiting on subprocesses to finish execution.
2026-07-17T15:37:00.130820267Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-17T15:37:00.131645337Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-17T15:37:00.132806017Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-17T15:37:00.134237200Z INFO: Collecting host service logs for crio
2026-07-17T15:37:00.134579928Z INFO: Collecting host service logs for kubelet
2026-07-17T15:37:00.134932757Z INFO: Collecting host service logs for rpm-ostreed
2026-07-17T15:37:00.135253853Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-17T15:37:00.135625196Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-17T15:37:00.135946208Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-17T15:37:00.136277575Z INFO: Collecting host service logs for NetworkManager
2026-07-17T15:37:00.136624169Z INFO: Collecting host service logs for openvswitch
2026-07-17T15:37:00.136963131Z INFO: Collecting host service logs for ovs-configuration
2026-07-17T15:37:00.137307224Z INFO: Collecting host service logs for ovsdb-server
2026-07-17T15:37:00.137668146Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-17T15:37:00.138075093Z INFO: Waiting for worker host service log collection to complete ...
2026-07-17T15:37:00.144767769Z Executing Istio gather script
2026-07-17T15:37:00.157105226Z INFO: Gathering HAProxy config files
2026-07-17T15:37:00.343320985Z INFO: Waiting for node performance related collection to complete ...
2026-07-17T15:37:00.753371431Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-17T15:37:00.829972146Z No resources found in openshift-etcd namespace.
2026-07-17T15:37:00.892271087Z Wrote inspect data to must-gather.
2026-07-17T15:37:00.961130900Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-17T15:37:00.961166538Z INFO: on-disk MachineConfig config collection complete.
2026-07-17T15:37:01.156347073Z INFO: "metallb-operator" not detected. Skipping.
2026-07-17T15:37:01.165269110Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-17T15:37:01.270086410Z No resources found
2026-07-17T15:37:01.314791321Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-17T15:37:01.334883019Z ERROR: No running kube-apiserver pods found
2026-07-17T15:37:01.467871728Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-cpnps
2026-07-17T15:37:01.467924742Z insights-runtime-extractor-fvvpx
2026-07-17T15:37:01.467924742Z insights-runtime-extractor-nvh89
2026-07-17T15:37:01.467924742Z insights-runtime-extractor-t6g87
2026-07-17T15:37:01.512192649Z Wrote inspect data to must-gather.
2026-07-17T15:37:01.548339732Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-17T15:37:01.559084400Z error: the server doesn't have a resource type "performanceprofile"
2026-07-17T15:37:01.697207054Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-17T15:37:01.865245447Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-17T15:37:01.865245447Z See 'oc rsync -h' for help and examples
2026-07-17T15:37:01.956322132Z tar: Removing leading `/' from member names
2026-07-17T15:37:01.961934330Z INFO: HAProxy config collection complete.
2026-07-17T15:37:02.139087797Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-17T15:37:02.188180004Z INFO: Worker host service log collection to complete.
2026-07-17T15:37:02.214052888Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-17T15:37:02.214917046Z Wrote inspect data to must-gather.
2026-07-17T15:37:02.331422103Z Wrote inspect data to must-gather.
2026-07-17T15:37:02.377821547Z Wrote inspect data to must-gather.
2026-07-17T15:37:02.411316065Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-17T15:37:02.468068366Z No resources found
2026-07-17T15:37:02.478509375Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-17T15:37:02.554759128Z Wrote inspect data to must-gather.
2026-07-17T15:37:02.620750063Z Wrote inspect data to must-gather.
2026-07-17T15:37:02.663172779Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-17T15:37:02.680201177Z INFO: 'previous-logs' folder not found on ip-10-0-129-9.ec2.internal, skipping...
2026-07-17T15:37:02.730749989Z Gathering data for ns/openshift-monitoring...
2026-07-17T15:37:02.759609259Z Wrote inspect data to must-gather.
2026-07-17T15:37:02.810207373Z error: the server doesn't have a resource type "machineconfigs"
2026-07-17T15:37:02.885541972Z INFO: Getting rules from prometheus-k8s-0
2026-07-17T15:37:02.975866695Z INFO: 'previous-logs' folder not found on ip-10-0-131-245.ec2.internal, skipping...
2026-07-17T15:37:02.988053580Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-17T15:37:03.039655448Z Wrote inspect data to must-gather.
2026-07-17T15:37:03.245080291Z Wrote inspect data to must-gather.
2026-07-17T15:37:03.245631277Z INFO: 'previous-logs' folder not found on ip-10-0-134-254.ec2.internal, skipping...
2026-07-17T15:37:03.245657182Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-17T15:37:03.245683703Z INFO: Machine Config Daemon termination log collection complete.
2026-07-17T15:37:03.312628588Z Wrote inspect data to must-gather.
2026-07-17T15:37:03.323617456Z INFO: Getting status/config from prometheus-k8s-0
2026-07-17T15:37:03.412211872Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-17T15:37:03.590705436Z INFO: INTERCONNECT MODE
2026-07-17T15:37:03.590739719Z INFO: Gathering ovn-kubernetes DBs
2026-07-17T15:37:03.636564908Z Wrote inspect data to must-gather.
2026-07-17T15:37:03.670360942Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-17T15:37:03.748085352Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-17T15:37:03.763214889Z INFO: Gathering OVN_Northbound from ovnkube-node-9ntf7...
2026-07-17T15:37:03.763502826Z INFO: Gathering OVN_Northbound from ovnkube-node-v4c8p...
2026-07-17T15:37:03.763737369Z INFO: Gathering OVN_Northbound from ovnkube-node-vmc5m...
2026-07-17T15:37:03.764002182Z INFO: Gathering OVN_Southbound from ovnkube-node-9ntf7...
2026-07-17T15:37:03.764264789Z INFO: Gathering OVN_Southbound from ovnkube-node-v4c8p...
2026-07-17T15:37:03.764519669Z INFO: Gathering OVN_Southbound from ovnkube-node-vmc5m...
2026-07-17T15:37:03.899625502Z Wrote inspect data to must-gather.
2026-07-17T15:37:03.969204424Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:04.053776039Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-17T15:37:04.068643171Z tar: Removing leading `/' from member names
2026-07-17T15:37:04.086164088Z tar: Removing leading `/' from member names
2026-07-17T15:37:04.107581520Z tar: Removing leading `/' from member names
2026-07-17T15:37:04.111216606Z tar: Removing leading `/' from member names
2026-07-17T15:37:04.136448708Z tar: Removing leading `/' from member names
2026-07-17T15:37:04.136713863Z tar: Removing leading `/' from member names
2026-07-17T15:37:04.209204255Z Wrote inspect data to must-gather.
2026-07-17T15:37:04.232743115Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:04.235564297Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-17T15:37:04.316594778Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-17T15:37:04.465424481Z Wrote inspect data to must-gather.
2026-07-17T15:37:04.476946718Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:04.560772741Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-17T15:37:04.577627473Z INFO: Waiting for network log collection to complete ...
2026-07-17T15:37:04.577676794Z INFO: Waiting for ovnk database copies to complete ...
2026-07-17T15:37:04.578167779Z INFO: Copying ovnk databases complete.
2026-07-17T15:37:04.579854402Z 2.3M	must-gather/network_logs/ovnk_database_store
2026-07-17T15:37:04.582431589Z ovnk_database_store/
2026-07-17T15:37:04.582510852Z ovnk_database_store/ovnkube-node-vmc5m_sbdb
2026-07-17T15:37:04.601749909Z ovnk_database_store/ovnkube-node-vmc5m_nbdb
2026-07-17T15:37:04.603469348Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-17T15:37:04.606820527Z ovnk_database_store/ovnkube-node-v4c8p_nbdb
2026-07-17T15:37:04.613325779Z ovnk_database_store/ovnkube-node-v4c8p_sbdb
2026-07-17T15:37:04.629769672Z ovnk_database_store/ovnkube-node-9ntf7_nbdb
2026-07-17T15:37:04.636559549Z ovnk_database_store/ovnkube-node-9ntf7_sbdb
2026-07-17T15:37:04.657774640Z INFO: Network log collection complete.
2026-07-17T15:37:04.679911523Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:04.683035947Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-17T15:37:04.699597257Z Wrote inspect data to must-gather.
2026-07-17T15:37:04.872295591Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:04.908526227Z Wrote inspect data to must-gather.
2026-07-17T15:37:04.973372240Z INFO: Getting status from alertmanager-main-0
2026-07-17T15:37:05.068982112Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:05.071956061Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-17T15:37:05.116946819Z W0717 15:37:05.116894    1575 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-17T15:37:05.122070873Z Wrote inspect data to must-gather.
2026-07-17T15:37:05.172416396Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-17T15:37:05.309171959Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:05.347900245Z Gathering data for ns/openshift-network-console...
2026-07-17T15:37:05.387586770Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-17T15:37:05.390412626Z INFO: Fallback to identify the container image from release info
2026-07-17T15:37:05.507640054Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:05.510216736Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-17T15:37:05.671030414Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:05.762341020Z Gathering data for ns/openshift-console-operator...
2026-07-17T15:37:05.862135643Z Wrote inspect data to /must-gather/istio.
2026-07-17T15:37:06.000996234Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-17T15:37:06.139596396Z daemonset.apps/perf-node-gather-daemonset created
2026-07-17T15:37:06.147767984Z Gathering data for ns/openshift-console...
2026-07-17T15:37:06.279903864Z Waiting for performance profile collector pods to become ready: 1
2026-07-17T15:37:06.552013716Z Done executing Istio gather script
2026-07-17T15:37:06.667958284Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-17T15:37:07.115870996Z Gathering data for ns/openshift-dns-operator...
2026-07-17T15:37:07.379660730Z Gathering data for ns/openshift-dns...
2026-07-17T15:37:07.400761269Z Waiting for performance profile collector pods to become ready: 2
2026-07-17T15:37:07.667994291Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-17T15:37:07.969746236Z Gathering data for ns/openshift-image-registry...
2026-07-17T15:37:08.445288508Z Gathering data for ns/openshift-ingress-operator...
2026-07-17T15:37:08.528529065Z Waiting for performance profile collector pods to become ready: 3
2026-07-17T15:37:08.776474496Z Gathering data for ns/openshift-ingress...
2026-07-17T15:37:09.130081244Z Gathering data for ns/openshift-ingress-canary...
2026-07-17T15:37:09.504841174Z Gathering data for ns/openshift-insights...
2026-07-17T15:37:09.633895771Z Waiting for performance profile collector pods to become ready: 4
2026-07-17T15:37:10.745127425Z Waiting for performance profile collector pods to become ready: 5
2026-07-17T15:37:11.319669960Z Gathering data for ns/openshift-config...
2026-07-17T15:37:11.584283066Z Gathering data for ns/openshift-config-managed...
2026-07-17T15:37:11.848178767Z Waiting for performance profile collector pods to become ready: 6
2026-07-17T15:37:11.983655024Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-17T15:37:12.230137060Z Gathering data for ns/openshift-kube-apiserver...
2026-07-17T15:37:12.469513495Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-17T15:37:12.674268658Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-17T15:37:12.721240810Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-17T15:37:12.971475098Z Waiting for performance profile collector pods to become ready: 7
2026-07-17T15:37:13.002999083Z Gathering data for ns/openshift-kube-scheduler...
2026-07-17T15:37:13.307622567Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-17T15:37:13.567583385Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-17T15:37:13.866605768Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-17T15:37:14.085486420Z Waiting for performance profile collector pods to become ready: 8
2026-07-17T15:37:14.151909833Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-17T15:37:14.682292210Z Gathering data for ns/openshift-multus...
2026-07-17T15:37:15.188837770Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-17T15:37:15.427824670Z Collecting performance related data for node ip-10-0-131-245.ec2.internal
2026-07-17T15:37:15.434748595Z Collecting performance related data for node ip-10-0-134-254.ec2.internal
2026-07-17T15:37:15.442107777Z Collecting performance related data for node ip-10-0-129-9.ec2.internal
2026-07-17T15:37:16.068097343Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-17T15:37:17.287681085Z Gathering data for ns/openshift-host-network...
2026-07-17T15:37:17.654130433Z Gathering data for ns/openshift-network-diagnostics...
2026-07-17T15:37:17.680053422Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-17T15:37:18.193792196Z Gathering data for ns/openshift-network-node-identity...
2026-07-17T15:37:18.708009303Z Gathering data for ns/openshift-network-operator...
2026-07-17T15:37:19.079899969Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-17T15:37:19.330940235Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-17T15:37:19.815329644Z Gathering data for ns/openshift-apiserver-operator...
2026-07-17T15:37:20.151908765Z Gathering data for ns/openshift-apiserver...
2026-07-17T15:37:20.530831240Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-17T15:37:20.785269288Z Gathering data for ns/openshift-controller-manager...
2026-07-17T15:37:21.046000434Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-17T15:37:21.669382079Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-17T15:37:21.944724187Z Gathering data for ns/openshift-service-ca-operator...
2026-07-17T15:37:22.231439218Z Gathering data for ns/openshift-service-ca...
2026-07-17T15:37:22.591554329Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-17T15:37:22.684596796Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-17T15:37:23.435673402Z Wrote inspect data to must-gather.
2026-07-17T15:37:23.435712904Z error: inspection completed with the errors occurred while gathering data:
2026-07-17T15:37:23.435712904Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-17T15:37:23.646609232Z Collecting kubelet logs for node ip-10-0-131-245.ec2.internal
2026-07-17T15:37:23.648328880Z Collecting kubelet logs for node ip-10-0-134-254.ec2.internal
2026-07-17T15:37:23.650486674Z Collecting kubelet logs for node ip-10-0-129-9.ec2.internal
2026-07-17T15:37:23.943019456Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-mp64x namespace
2026-07-17T15:37:23.947837663Z INFO: Node performance data collection complete.
2026-07-17T15:37:24.068223332Z error: the server doesn't have a resource type "clusters"
2026-07-17T15:37:24.222092604Z Caches written to disk