2026-06-05T20:41:48.104455012Z [disk usage checker] Started
2026-06-05T20:41:48.108010356Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-05T20:41:48.298462963Z Gathering data for ns/openshift-cluster-version...
2026-06-05T20:41:48.421819091Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T20:41:48.493719518Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T20:41:48.566760064Z Gathering data for ns/default...
2026-06-05T20:41:48.844786076Z Gathering data for ns/openshift...
2026-06-05T20:41:49.226647806Z Gathering data for ns/kube-system...
2026-06-05T20:41:49.878235347Z Gathering data for ns/openshift-etcd...
2026-06-05T20:41:50.142291434Z Wrote inspect data to must-gather.
2026-06-05T20:41:50.142322759Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T20:41:50.142322759Z     namespaces "assisted-installer" not found
2026-06-05T20:41:50.560781855Z Waiting on subprocesses to finish execution.
2026-06-05T20:41:50.563831525Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-05T20:41:50.567668169Z INFO: Collecting host service logs for crio
2026-06-05T20:41:50.567946153Z INFO: Collecting host service logs for kubelet
2026-06-05T20:41:50.568249155Z INFO: Collecting host service logs for rpm-ostreed
2026-06-05T20:41:50.568609755Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-05T20:41:50.568977348Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-05T20:41:50.569492628Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-05T20:41:50.569806493Z INFO: Collecting host service logs for NetworkManager
2026-06-05T20:41:50.570084544Z INFO: Collecting host service logs for openvswitch
2026-06-05T20:41:50.570372023Z INFO: Collecting host service logs for ovs-configuration
2026-06-05T20:41:50.570821640Z INFO: Collecting host service logs for ovsdb-server
2026-06-05T20:41:50.571174438Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-05T20:41:50.571559118Z INFO: Waiting for worker host service log collection to complete ...
2026-06-05T20:41:50.572107521Z Executing Istio gather script
2026-06-05T20:41:50.577233193Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-05T20:41:50.585419926Z INFO: Gathering HAProxy config files
2026-06-05T20:41:50.640521300Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-05T20:41:50.925279581Z INFO: "metallb-operator" not detected. Skipping.
2026-06-05T20:41:50.947796053Z INFO: Waiting for node performance related collection to complete ...
2026-06-05T20:41:51.224612953Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-05T20:41:51.461650150Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-05T20:41:51.491281706Z No resources found
2026-06-05T20:41:51.565914051Z INFO: Collecting Insights Archives from insights-operator-896549976-txs7k
2026-06-05T20:41:51.565997314Z insights-runtime-extractor-9bl92
2026-06-05T20:41:51.566019925Z insights-runtime-extractor-jtpd9
2026-06-05T20:41:51.571602400Z insights-runtime-extractor-ncc8t
2026-06-05T20:41:51.571924156Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-05T20:41:51.576096205Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-05T20:41:51.632584854Z No resources found in openshift-etcd namespace.
2026-06-05T20:41:51.637241185Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-05T20:41:51.637313007Z INFO: on-disk MachineConfig config collection complete.
2026-06-05T20:41:51.657143537Z ERROR: No running kube-apiserver pods found
2026-06-05T20:41:51.772590891Z error: the server doesn't have a resource type "performanceprofile"
2026-06-05T20:41:51.915892118Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-05T20:41:51.915892118Z See 'oc rsync -h' for help and examples
2026-06-05T20:41:51.999967587Z INFO: Worker host service log collection to complete.
2026-06-05T20:41:52.007603298Z Wrote inspect data to must-gather.
2026-06-05T20:41:52.169005572Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-05T20:41:52.179432958Z Wrote inspect data to must-gather.
2026-06-05T20:41:52.602837263Z Wrote inspect data to must-gather.
2026-06-05T20:41:52.735285032Z Wrote inspect data to must-gather.
2026-06-05T20:41:52.757126329Z Gathering data for ns/openshift-monitoring...
2026-06-05T20:41:52.775338777Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-05T20:41:52.827584280Z Wrote inspect data to must-gather.
2026-06-05T20:41:52.845040055Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T20:41:52.979155193Z No resources found
2026-06-05T20:41:53.002737690Z INFO: 'previous-logs' folder not found on ip-10-0-133-23.ec2.internal, skipping...
2026-06-05T20:41:53.036946298Z Wrote inspect data to must-gather.
2026-06-05T20:41:53.067042733Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-05T20:41:53.115043160Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-05T20:41:53.138994670Z error: the server doesn't have a resource type "machineconfigs"
2026-06-05T20:41:53.189759984Z Wrote inspect data to must-gather.
2026-06-05T20:41:53.212521898Z tar: Removing leading `/' from member names
2026-06-05T20:41:53.220991085Z INFO: HAProxy config collection complete.
2026-06-05T20:41:53.237315154Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-05T20:41:53.315379302Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-05T20:41:53.358790100Z INFO: 'previous-logs' folder not found on ip-10-0-134-205.ec2.internal, skipping...
2026-06-05T20:41:53.364410197Z INFO: Getting rules from prometheus-k8s-0
2026-06-05T20:41:53.458414282Z Wrote inspect data to must-gather.
2026-06-05T20:41:53.546028883Z Wrote inspect data to must-gather.
2026-06-05T20:41:53.640747012Z INFO: 'previous-logs' folder not found on ip-10-0-143-107.ec2.internal, skipping...
2026-06-05T20:41:53.640804575Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-05T20:41:53.640847566Z INFO: Machine Config Daemon termination log collection complete.
2026-06-05T20:41:53.708366321Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-05T20:41:53.716670426Z Wrote inspect data to must-gather.
2026-06-05T20:41:53.771512112Z INFO: Getting status/config from prometheus-k8s-0
2026-06-05T20:41:53.920066683Z Wrote inspect data to must-gather.
2026-06-05T20:41:53.965328378Z Wrote inspect data to must-gather.
2026-06-05T20:41:54.025190945Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-05T20:41:54.134196460Z Wrote inspect data to must-gather.
2026-06-05T20:41:54.188501509Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-05T20:41:54.239420272Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T20:41:54.241655618Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:54.274046706Z INFO: INTERCONNECT MODE
2026-06-05T20:41:54.274087604Z INFO: Gathering ovn-kubernetes DBs
2026-06-05T20:41:54.363959589Z Wrote inspect data to must-gather.
2026-06-05T20:41:54.433738188Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-05T20:41:54.434828436Z INFO: Gathering OVN_Northbound from ovnkube-node-bwdnw...
2026-06-05T20:41:54.435095836Z INFO: Gathering OVN_Northbound from ovnkube-node-h268z...
2026-06-05T20:41:54.435328030Z INFO: Gathering OVN_Northbound from ovnkube-node-ptlgq...
2026-06-05T20:41:54.435628813Z INFO: Gathering OVN_Southbound from ovnkube-node-bwdnw...
2026-06-05T20:41:54.436040189Z INFO: Gathering OVN_Southbound from ovnkube-node-h268z...
2026-06-05T20:41:54.436370151Z INFO: Gathering OVN_Southbound from ovnkube-node-ptlgq...
2026-06-05T20:41:54.483843948Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:54.488399853Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-05T20:41:54.701269453Z Wrote inspect data to must-gather.
2026-06-05T20:41:54.816441235Z tar: Removing leading `/' from member names
2026-06-05T20:41:54.821633642Z tar: Removing leading `/' from member names
2026-06-05T20:41:54.833133409Z tar: Removing leading `/' from member names
2026-06-05T20:41:54.835785860Z tar: Removing leading `/' from member names
2026-06-05T20:41:54.838994171Z tar: Removing leading `/' from member names
2026-06-05T20:41:54.849969485Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-05T20:41:54.850521628Z tar: Removing leading `/' from member names
2026-06-05T20:41:54.906199811Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:54.986459442Z W0605 20:41:54.986408    1439 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-05T20:41:54.993906573Z Wrote inspect data to must-gather.
2026-06-05T20:41:55.058969078Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T20:41:55.110119028Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-05T20:41:55.154093148Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:55.157154363Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-05T20:41:55.273879292Z Gathering data for ns/openshift-network-console...
2026-06-05T20:41:55.338449031Z INFO: Waiting for network log collection to complete ...
2026-06-05T20:41:55.338498436Z INFO: Waiting for ovnk database copies to complete ...
2026-06-05T20:41:55.339027957Z INFO: Copying ovnk databases complete.
2026-06-05T20:41:55.341227642Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-06-05T20:41:55.344926052Z ovnk_database_store/
2026-06-05T20:41:55.344975472Z ovnk_database_store/ovnkube-node-bwdnw_nbdb
2026-06-05T20:41:55.346360871Z Wrote inspect data to must-gather.
2026-06-05T20:41:55.356803830Z ovnk_database_store/ovnkube-node-ptlgq_sbdb
2026-06-05T20:41:55.364563747Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:55.375482872Z ovnk_database_store/ovnkube-node-bwdnw_sbdb
2026-06-05T20:41:55.399586352Z ovnk_database_store/ovnkube-node-h268z_sbdb
2026-06-05T20:41:55.416480199Z ovnk_database_store/ovnkube-node-ptlgq_nbdb
2026-06-05T20:41:55.423761333Z ovnk_database_store/ovnkube-node-h268z_nbdb
2026-06-05T20:41:55.435097516Z INFO: Network log collection complete.
2026-06-05T20:41:55.533465750Z INFO: Getting status from alertmanager-main-0
2026-06-05T20:41:55.582894565Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:55.586372086Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-05T20:41:55.667219269Z Wrote inspect data to must-gather.
2026-06-05T20:41:55.750538873Z Gathering data for ns/openshift-console-operator...
2026-06-05T20:41:55.825391176Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:55.930019056Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-05T20:41:55.932092707Z INFO: Fallback to identify the container image from release info
2026-06-05T20:41:56.022375612Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:56.025256645Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-05T20:41:56.143250751Z Gathering data for ns/openshift-console...
2026-06-05T20:41:56.178611449Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:56.347025147Z Wrote inspect data to /must-gather/istio.
2026-06-05T20:41:56.615414361Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-05T20:41:56.731514840Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-05T20:41:56.854660613Z daemonset.apps/perf-node-gather-daemonset created
2026-06-05T20:41:56.999631749Z Waiting for performance profile collector pods to become ready: 1
2026-06-05T20:41:57.028621422Z Done executing Istio gather script
2026-06-05T20:41:57.119471954Z Gathering data for ns/openshift-dns-operator...
2026-06-05T20:41:57.390982163Z Gathering data for ns/openshift-dns...
2026-06-05T20:41:57.921335101Z Gathering data for ns/openshift-image-registry...
2026-06-05T20:41:58.122599707Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-05T20:41:58.122815881Z Waiting for performance profile collector pods to become ready: 2
2026-06-05T20:41:58.356610044Z Gathering data for ns/openshift-ingress-operator...
2026-06-05T20:41:58.629382139Z Gathering data for ns/openshift-ingress...
2026-06-05T20:41:58.933112428Z Gathering data for ns/openshift-ingress-canary...
2026-06-05T20:41:59.241095722Z Waiting for performance profile collector pods to become ready: 3
2026-06-05T20:41:59.291928569Z Gathering data for ns/openshift-insights...
2026-06-05T20:42:00.343705230Z Waiting for performance profile collector pods to become ready: 4
2026-06-05T20:42:01.033194442Z Gathering data for ns/openshift-config...
2026-06-05T20:42:01.306803158Z Gathering data for ns/openshift-config-managed...
2026-06-05T20:42:01.451393586Z Waiting for performance profile collector pods to become ready: 5
2026-06-05T20:42:01.706989798Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-05T20:42:01.942965741Z Gathering data for ns/openshift-kube-apiserver...
2026-06-05T20:42:02.168170475Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-05T20:42:02.410063798Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-05T20:42:02.561649555Z Waiting for performance profile collector pods to become ready: 6
2026-06-05T20:42:02.667126919Z Gathering data for ns/openshift-kube-scheduler...
2026-06-05T20:42:02.897577020Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-05T20:42:03.127177564Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-05T20:42:03.147428489Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-05T20:42:03.420874245Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-05T20:42:03.681560971Z Waiting for performance profile collector pods to become ready: 7
2026-06-05T20:42:03.705012908Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-05T20:42:04.136865061Z Gathering data for ns/openshift-multus...
2026-06-05T20:42:04.778845702Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-05T20:42:05.015853704Z Collecting performance related data for node ip-10-0-134-205.ec2.internal
2026-06-05T20:42:05.023429753Z Collecting performance related data for node ip-10-0-133-23.ec2.internal
2026-06-05T20:42:05.033133380Z Collecting performance related data for node ip-10-0-143-107.ec2.internal
2026-06-05T20:42:05.543874454Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-05T20:42:06.832851461Z Gathering data for ns/openshift-host-network...
2026-06-05T20:42:07.124822436Z Gathering data for ns/openshift-network-diagnostics...
2026-06-05T20:42:07.659929370Z Gathering data for ns/openshift-network-node-identity...
2026-06-05T20:42:08.042985304Z Gathering data for ns/openshift-network-operator...
2026-06-05T20:42:08.132717066Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-05T20:42:08.398045203Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-05T20:42:08.621999606Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-05T20:42:09.069171183Z Gathering data for ns/openshift-apiserver-operator...
2026-06-05T20:42:09.332394804Z Gathering data for ns/openshift-apiserver...
2026-06-05T20:42:09.688551820Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-05T20:42:09.920689736Z Gathering data for ns/openshift-controller-manager...
2026-06-05T20:42:10.178377604Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-05T20:42:10.785771166Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-05T20:42:11.044120797Z Gathering data for ns/openshift-service-ca-operator...
2026-06-05T20:42:11.331138123Z Gathering data for ns/openshift-service-ca...
2026-06-05T20:42:11.643743682Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-05T20:42:12.465196762Z Wrote inspect data to must-gather.
2026-06-05T20:42:12.465238330Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T20:42:12.465238330Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-05T20:42:13.073396766Z Collecting kubelet logs for node ip-10-0-134-205.ec2.internal
2026-06-05T20:42:13.075157433Z Collecting kubelet logs for node ip-10-0-133-23.ec2.internal
2026-06-05T20:42:13.076831551Z Collecting kubelet logs for node ip-10-0-143-107.ec2.internal
2026-06-05T20:42:13.139254954Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-06-05T20:42:13.369534698Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ctcx5 namespace
2026-06-05T20:42:13.374261343Z INFO: Node performance data collection complete.
2026-06-05T20:42:13.500483411Z error: the server doesn't have a resource type "clusters"
2026-06-05T20:42:13.662738762Z Caches written to disk