2026-07-21T08:03:04.091928807Z [disk usage checker] Started
2026-07-21T08:03:04.095817055Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T08:03:04.293250917Z Gathering data for ns/openshift-cluster-version...
2026-07-21T08:03:04.432027928Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-21T08:03:04.509657471Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T08:03:04.585880029Z Gathering data for ns/default...
2026-07-21T08:03:04.869487585Z Gathering data for ns/openshift...
2026-07-21T08:03:05.260458348Z Gathering data for ns/kube-system...
2026-07-21T08:03:05.934649261Z Gathering data for ns/openshift-etcd...
2026-07-21T08:03:06.212092821Z Wrote inspect data to must-gather.
2026-07-21T08:03:06.212127472Z error: inspection completed with the errors occurred while gathering data:
2026-07-21T08:03:06.212127472Z     namespaces "assisted-installer" not found
2026-07-21T08:03:06.661134736Z Waiting on subprocesses to finish execution.
2026-07-21T08:03:06.669655432Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-21T08:03:06.669983141Z INFO: Collecting host service logs for crio
2026-07-21T08:03:06.670288737Z INFO: Collecting host service logs for kubelet
2026-07-21T08:03:06.670638849Z INFO: Collecting host service logs for rpm-ostreed
2026-07-21T08:03:06.670981122Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-21T08:03:06.671328141Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-21T08:03:06.671533292Z INFO: Gathering HAProxy config files
2026-07-21T08:03:06.671684378Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-21T08:03:06.671992371Z INFO: Collecting host service logs for NetworkManager
2026-07-21T08:03:06.672312599Z INFO: Collecting host service logs for openvswitch
2026-07-21T08:03:06.672644367Z INFO: Collecting host service logs for ovs-configuration
2026-07-21T08:03:06.673063351Z INFO: Collecting host service logs for ovsdb-server
2026-07-21T08:03:06.673388167Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-21T08:03:06.673845919Z INFO: Waiting for worker host service log collection to complete ...
2026-07-21T08:03:06.674710088Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-21T08:03:06.689988398Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-21T08:03:06.701988120Z Executing Istio gather script
2026-07-21T08:03:06.716923129Z INFO: Waiting for node performance related collection to complete ...
2026-07-21T08:03:07.303393585Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-21T08:03:07.303477007Z INFO: on-disk MachineConfig config collection complete.
2026-07-21T08:03:07.404717318Z INFO: "metallb-operator" not detected. Skipping.
2026-07-21T08:03:07.502893039Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-21T08:03:07.582266769Z Wrote inspect data to must-gather.
2026-07-21T08:03:07.589346561Z No resources found in openshift-etcd namespace.
2026-07-21T08:03:07.589857331Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-21T08:03:07.816256958Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-21T08:03:07.913730417Z No resources found
2026-07-21T08:03:07.916718324Z INFO: Worker host service log collection to complete.
2026-07-21T08:03:07.983256404Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-21T08:03:08.002521410Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-pf2zq
2026-07-21T08:03:08.002570662Z insights-runtime-extractor-7hllx
2026-07-21T08:03:08.002591389Z insights-runtime-extractor-p7j7s
2026-07-21T08:03:08.002610950Z insights-runtime-extractor-ph28w
2026-07-21T08:03:08.004452611Z error: the server doesn't have a resource type "performanceprofile"
2026-07-21T08:03:08.067255110Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-21T08:03:08.495797111Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-21T08:03:08.761452033Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-21T08:03:08.761452033Z See 'oc rsync -h' for help and examples
2026-07-21T08:03:08.780275950Z Wrote inspect data to must-gather.
2026-07-21T08:03:08.781454643Z ERROR: No running kube-apiserver pods found
2026-07-21T08:03:08.830216819Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-21T08:03:08.837635167Z Wrote inspect data to must-gather.
2026-07-21T08:03:08.873630972Z No resources found
2026-07-21T08:03:08.919165361Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T08:03:08.989189320Z Wrote inspect data to must-gather.
2026-07-21T08:03:09.044579561Z Wrote inspect data to must-gather.
2026-07-21T08:03:09.063390682Z Wrote inspect data to must-gather.
2026-07-21T08:03:09.115911900Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T08:03:09.151774090Z Gathering data for ns/openshift-monitoring...
2026-07-21T08:03:09.182664712Z Wrote inspect data to must-gather.
2026-07-21T08:03:09.196880919Z tar: Removing leading `/' from member names
2026-07-21T08:03:09.205789670Z INFO: 'previous-logs' folder not found on ip-10-0-130-92.ec2.internal, skipping...
2026-07-21T08:03:09.206819359Z INFO: HAProxy config collection complete.
2026-07-21T08:03:09.286252929Z error: the server doesn't have a resource type "machineconfigs"
2026-07-21T08:03:09.393919517Z INFO: Getting rules from prometheus-k8s-0
2026-07-21T08:03:09.488812882Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-21T08:03:09.506213839Z Wrote inspect data to must-gather.
2026-07-21T08:03:09.539361348Z INFO: 'previous-logs' folder not found on ip-10-0-131-166.ec2.internal, skipping...
2026-07-21T08:03:09.757811260Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-21T08:03:09.766324896Z Wrote inspect data to must-gather.
2026-07-21T08:03:09.809527746Z Wrote inspect data to must-gather.
2026-07-21T08:03:09.846009743Z INFO: 'previous-logs' folder not found on ip-10-0-131-182.ec2.internal, skipping...
2026-07-21T08:03:09.846048131Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-21T08:03:09.846083837Z INFO: Machine Config Daemon termination log collection complete.
2026-07-21T08:03:09.874599520Z INFO: Getting status/config from prometheus-k8s-0
2026-07-21T08:03:09.941254949Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-21T08:03:10.074577613Z Wrote inspect data to must-gather.
2026-07-21T08:03:10.158110639Z Wrote inspect data to must-gather.
2026-07-21T08:03:10.295584176Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-21T08:03:10.363895452Z INFO: INTERCONNECT MODE
2026-07-21T08:03:10.363895452Z INFO: Gathering ovn-kubernetes DBs
2026-07-21T08:03:10.370871398Z Wrote inspect data to must-gather.
2026-07-21T08:03:10.536982323Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-21T08:03:10.539729277Z INFO: Gathering OVN_Northbound from ovnkube-node-b5nxg...
2026-07-21T08:03:10.539954030Z INFO: Gathering OVN_Northbound from ovnkube-node-cxknp...
2026-07-21T08:03:10.540209987Z INFO: Gathering OVN_Northbound from ovnkube-node-mrzh9...
2026-07-21T08:03:10.540519070Z INFO: Gathering OVN_Southbound from ovnkube-node-b5nxg...
2026-07-21T08:03:10.540851898Z INFO: Gathering OVN_Southbound from ovnkube-node-cxknp...
2026-07-21T08:03:10.541143146Z INFO: Gathering OVN_Southbound from ovnkube-node-mrzh9...
2026-07-21T08:03:10.557791675Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-21T08:03:10.621930305Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-21T08:03:10.651402973Z Wrote inspect data to must-gather.
2026-07-21T08:03:10.828252654Z tar: Removing leading `/' from member names
2026-07-21T08:03:10.858591345Z tar: Removing leading `/' from member names
2026-07-21T08:03:10.877442380Z tar: Removing leading `/' from member names
2026-07-21T08:03:10.894274825Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-21T08:03:10.898325777Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:10.912663066Z tar: Removing leading `/' from member names
2026-07-21T08:03:10.956560295Z tar: Removing leading `/' from member names
2026-07-21T08:03:10.958045593Z tar: Removing leading `/' from member names
2026-07-21T08:03:11.039560882Z Wrote inspect data to must-gather.
2026-07-21T08:03:11.162281598Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:11.166339325Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-21T08:03:11.169724179Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-21T08:03:11.294289921Z W0721 08:03:11.294240    1435 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-21T08:03:11.301817587Z Wrote inspect data to must-gather.
2026-07-21T08:03:11.398618281Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-21T08:03:11.408427632Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:11.424298072Z INFO: Waiting for network log collection to complete ...
2026-07-21T08:03:11.424347173Z INFO: Waiting for ovnk database copies to complete ...
2026-07-21T08:03:11.424885632Z INFO: Copying ovnk databases complete.
2026-07-21T08:03:11.426856631Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-21T08:03:11.430422153Z ovnk_database_store/
2026-07-21T08:03:11.430473212Z ovnk_database_store/ovnkube-node-b5nxg_sbdb
2026-07-21T08:03:11.453087940Z ovnk_database_store/ovnkube-node-b5nxg_nbdb
2026-07-21T08:03:11.459994475Z ovnk_database_store/ovnkube-node-cxknp_sbdb
2026-07-21T08:03:11.483028414Z ovnk_database_store/ovnkube-node-cxknp_nbdb
2026-07-21T08:03:11.490008668Z ovnk_database_store/ovnkube-node-mrzh9_nbdb
2026-07-21T08:03:11.495655937Z ovnk_database_store/ovnkube-node-mrzh9_sbdb
2026-07-21T08:03:11.516960914Z INFO: Network log collection complete.
2026-07-21T08:03:11.596483789Z Gathering data for ns/openshift-network-console...
2026-07-21T08:03:11.598959147Z INFO: Getting status from alertmanager-main-0
2026-07-21T08:03:11.644147696Z Wrote inspect data to must-gather.
2026-07-21T08:03:11.650166655Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:11.653034146Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-21T08:03:11.865183716Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:11.963276999Z Wrote inspect data to must-gather.
2026-07-21T08:03:12.063945388Z Gathering data for ns/openshift-console-operator...
2026-07-21T08:03:12.069225361Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:12.071934963Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-21T08:03:12.219386915Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-21T08:03:12.221381085Z INFO: Fallback to identify the container image from release info
2026-07-21T08:03:12.302193112Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:12.456225080Z Gathering data for ns/openshift-console...
2026-07-21T08:03:12.490419014Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:12.493432198Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-21T08:03:12.652018435Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:12.803977637Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-21T08:03:12.857988888Z Wrote inspect data to /must-gather/istio.
2026-07-21T08:03:12.944311071Z daemonset.apps/perf-node-gather-daemonset created
2026-07-21T08:03:12.992089774Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-21T08:03:13.094294480Z Waiting for performance profile collector pods to become ready: 1
2026-07-21T08:03:13.527431478Z Gathering data for ns/openshift-dns-operator...
2026-07-21T08:03:13.592523436Z Done executing Istio gather script
2026-07-21T08:03:13.819458401Z Gathering data for ns/openshift-dns...
2026-07-21T08:03:14.121865250Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T08:03:14.205804192Z Waiting for performance profile collector pods to become ready: 2
2026-07-21T08:03:14.381192207Z Gathering data for ns/openshift-image-registry...
2026-07-21T08:03:14.814826694Z Gathering data for ns/openshift-ingress-operator...
2026-07-21T08:03:15.107102467Z Gathering data for ns/openshift-ingress...
2026-07-21T08:03:15.316802691Z Waiting for performance profile collector pods to become ready: 3
2026-07-21T08:03:15.415330762Z Gathering data for ns/openshift-ingress-canary...
2026-07-21T08:03:15.761361843Z Gathering data for ns/openshift-insights...
2026-07-21T08:03:16.429512776Z Waiting for performance profile collector pods to become ready: 4
2026-07-21T08:03:17.548568487Z Waiting for performance profile collector pods to become ready: 5
2026-07-21T08:03:17.557272530Z Gathering data for ns/openshift-jobset-operator...
2026-07-21T08:03:18.318817585Z Gathering data for ns/openshift-config...
2026-07-21T08:03:18.563255927Z Gathering data for ns/openshift-config-managed...
2026-07-21T08:03:18.673970266Z Waiting for performance profile collector pods to become ready: 6
2026-07-21T08:03:18.971921344Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-21T08:03:19.126505646Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T08:03:19.251641168Z Gathering data for ns/openshift-kube-apiserver...
2026-07-21T08:03:19.487187420Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-21T08:03:19.771142081Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-21T08:03:19.800233991Z Waiting for performance profile collector pods to become ready: 7
2026-07-21T08:03:20.048635767Z Gathering data for ns/openshift-kube-scheduler...
2026-07-21T08:03:20.280438352Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-21T08:03:20.543779124Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-21T08:03:20.845936901Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-21T08:03:20.922153483Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-21T08:03:21.163533223Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-21T08:03:21.163754132Z Collecting performance related data for node ip-10-0-130-92.ec2.internal
2026-07-21T08:03:21.172495857Z Collecting performance related data for node ip-10-0-131-182.ec2.internal
2026-07-21T08:03:21.181065286Z Collecting performance related data for node ip-10-0-131-166.ec2.internal
2026-07-21T08:03:21.760758721Z Gathering data for ns/openshift-multus...
2026-07-21T08:03:23.165384709Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-21T08:03:24.132086256Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T08:03:24.411131449Z Gathering data for ns/openshift-host-network...
2026-07-21T08:03:24.698734625Z Gathering data for ns/openshift-network-diagnostics...
2026-07-21T08:03:25.218396136Z Gathering data for ns/openshift-network-node-identity...
2026-07-21T08:03:25.611683263Z Gathering data for ns/openshift-network-operator...
2026-07-21T08:03:25.983250891Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-21T08:03:26.234873091Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-21T08:03:26.708173308Z Gathering data for ns/openshift-apiserver-operator...
2026-07-21T08:03:26.994857241Z Gathering data for ns/openshift-apiserver...
2026-07-21T08:03:27.372670217Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-21T08:03:27.627990062Z Gathering data for ns/openshift-controller-manager...
2026-07-21T08:03:27.897127400Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-21T08:03:28.519147381Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-21T08:03:28.801944105Z Gathering data for ns/openshift-service-ca-operator...
2026-07-21T08:03:29.117475029Z Gathering data for ns/openshift-service-ca...
2026-07-21T08:03:29.137170569Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-21T08:03:29.247926388Z Collecting kubelet logs for node ip-10-0-130-92.ec2.internal
2026-07-21T08:03:29.249743181Z Collecting kubelet logs for node ip-10-0-131-182.ec2.internal
2026-07-21T08:03:29.251453854Z Collecting kubelet logs for node ip-10-0-131-166.ec2.internal
2026-07-21T08:03:29.500230877Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-21T08:03:29.586515517Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-flzk6 namespace
2026-07-21T08:03:29.591466982Z INFO: Node performance data collection complete.
2026-07-21T08:03:30.367818038Z Wrote inspect data to must-gather.
2026-07-21T08:03:30.367861178Z error: inspection completed with the errors occurred while gathering data:
2026-07-21T08:03:30.367861178Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-21T08:03:30.493210271Z error: the server doesn't have a resource type "clusters"
2026-07-21T08:03:30.638652575Z Caches written to disk