2026-07-01T17:17:47.585485520Z [disk usage checker] Started
2026-07-01T17:17:47.589599453Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-01T17:17:47.803873719Z Gathering data for ns/openshift-cluster-version...
2026-07-01T17:17:47.932903446Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-01T17:17:48.023517977Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-01T17:17:48.107042558Z Gathering data for ns/default...
2026-07-01T17:17:48.435861565Z Gathering data for ns/openshift...
2026-07-01T17:17:48.834574046Z Gathering data for ns/kube-system...
2026-07-01T17:17:49.556744433Z Gathering data for ns/openshift-etcd...
2026-07-01T17:17:49.857018207Z Wrote inspect data to must-gather.
2026-07-01T17:17:49.857054178Z error: inspection completed with the errors occurred while gathering data:
2026-07-01T17:17:49.857054178Z     namespaces "assisted-installer" not found
2026-07-01T17:17:50.208925077Z Waiting on subprocesses to finish execution.
2026-07-01T17:17:50.217661364Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-01T17:17:50.220536107Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-01T17:17:50.222948394Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-01T17:17:50.223814590Z INFO: Collecting host service logs for crio
2026-07-01T17:17:50.224141180Z INFO: Collecting host service logs for kubelet
2026-07-01T17:17:50.224471103Z INFO: Collecting host service logs for rpm-ostreed
2026-07-01T17:17:50.224816840Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-01T17:17:50.225150226Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-01T17:17:50.225472586Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-01T17:17:50.225871616Z INFO: Collecting host service logs for NetworkManager
2026-07-01T17:17:50.226188787Z INFO: Collecting host service logs for openvswitch
2026-07-01T17:17:50.226516321Z INFO: Collecting host service logs for ovs-configuration
2026-07-01T17:17:50.226861555Z INFO: Collecting host service logs for ovsdb-server
2026-07-01T17:17:50.227208379Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-01T17:17:50.227603977Z INFO: Waiting for worker host service log collection to complete ...
2026-07-01T17:17:50.231340728Z INFO: Gathering HAProxy config files
2026-07-01T17:17:50.235826561Z Executing Istio gather script
2026-07-01T17:17:50.260129081Z INFO: Waiting for node performance related collection to complete ...
2026-07-01T17:17:50.685950396Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-01T17:17:50.733500441Z INFO: Collecting Insights Archives from insights-operator-5f67b55555-r9ph4
2026-07-01T17:17:50.733570039Z insights-runtime-extractor-jzl28
2026-07-01T17:17:50.733591971Z insights-runtime-extractor-vxg6d
2026-07-01T17:17:50.733610905Z insights-runtime-extractor-w2cjg
2026-07-01T17:17:50.860160115Z Wrote inspect data to must-gather.
2026-07-01T17:17:50.951180873Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-01T17:17:51.103455192Z No resources found in openshift-etcd namespace.
2026-07-01T17:17:51.553094110Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-01T17:17:51.573625672Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-01T17:17:51.573625672Z See 'oc rsync -h' for help and examples
2026-07-01T17:17:51.622335138Z No resources found
2026-07-01T17:17:51.628946902Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-01T17:17:51.785933589Z error: the server doesn't have a resource type "performanceprofile"
2026-07-01T17:17:51.800189536Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-01T17:17:51.841115782Z INFO: Worker host service log collection to complete.
2026-07-01T17:17:51.861214102Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-01T17:17:51.861247210Z INFO: on-disk MachineConfig config collection complete.
2026-07-01T17:17:51.900311356Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-01T17:17:51.928570705Z Wrote inspect data to must-gather.
2026-07-01T17:17:52.133957840Z ERROR: No running kube-apiserver pods found
2026-07-01T17:17:52.176142348Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-01T17:17:52.309202101Z INFO: "metallb-operator" not detected. Skipping.
2026-07-01T17:17:52.350426544Z Wrote inspect data to must-gather.
2026-07-01T17:17:52.454504244Z Wrote inspect data to must-gather.
2026-07-01T17:17:52.508338005Z tar: Removing leading `/' from member names
2026-07-01T17:17:52.513793968Z INFO: HAProxy config collection complete.
2026-07-01T17:17:52.548663402Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-01T17:17:52.571361879Z No resources found
2026-07-01T17:17:52.589425152Z INFO: 'previous-logs' folder not found on ip-10-0-131-254.ec2.internal, skipping...
2026-07-01T17:17:52.614575978Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-01T17:17:52.776269165Z Wrote inspect data to must-gather.
2026-07-01T17:17:52.778971153Z Wrote inspect data to must-gather.
2026-07-01T17:17:52.795241805Z error: the server doesn't have a resource type "machineconfigs"
2026-07-01T17:17:52.837787066Z Wrote inspect data to must-gather.
2026-07-01T17:17:52.843292680Z Gathering data for ns/openshift-monitoring...
2026-07-01T17:17:52.995666701Z INFO: 'previous-logs' folder not found on ip-10-0-137-171.ec2.internal, skipping...
2026-07-01T17:17:53.015028058Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-01T17:17:53.025769802Z INFO: Getting rules from prometheus-k8s-0
2026-07-01T17:17:53.101632110Z Wrote inspect data to must-gather.
2026-07-01T17:17:53.178822346Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-01T17:17:53.275634982Z Wrote inspect data to must-gather.
2026-07-01T17:17:53.312199555Z INFO: 'previous-logs' folder not found on ip-10-0-139-65.ec2.internal, skipping...
2026-07-01T17:17:53.312234266Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-01T17:17:53.312260790Z INFO: Machine Config Daemon termination log collection complete.
2026-07-01T17:17:53.375036514Z Wrote inspect data to must-gather.
2026-07-01T17:17:53.447250676Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-01T17:17:53.454232309Z INFO: Getting status/config from prometheus-k8s-0
2026-07-01T17:17:53.631069775Z Wrote inspect data to must-gather.
2026-07-01T17:17:53.664896999Z Wrote inspect data to must-gather.
2026-07-01T17:17:53.724882987Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-01T17:17:53.859976949Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-01T17:17:53.874817613Z Wrote inspect data to must-gather.
2026-07-01T17:17:53.919189134Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:53.920809041Z INFO: INTERCONNECT MODE
2026-07-01T17:17:53.920839235Z INFO: Gathering ovn-kubernetes DBs
2026-07-01T17:17:54.083591873Z Wrote inspect data to must-gather.
2026-07-01T17:17:54.088508054Z INFO: Gathering OVN_Northbound from ovnkube-node-j4tb4...
2026-07-01T17:17:54.088845050Z INFO: Gathering OVN_Northbound from ovnkube-node-mvdwc...
2026-07-01T17:17:54.089126941Z INFO: Gathering OVN_Northbound from ovnkube-node-snn2l...
2026-07-01T17:17:54.089487206Z INFO: Gathering OVN_Southbound from ovnkube-node-j4tb4...
2026-07-01T17:17:54.089972873Z INFO: Gathering OVN_Southbound from ovnkube-node-mvdwc...
2026-07-01T17:17:54.090273334Z INFO: Gathering OVN_Southbound from ovnkube-node-snn2l...
2026-07-01T17:17:54.141110973Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-01T17:17:54.189630756Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:54.193063399Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-01T17:17:54.428184455Z tar: Removing leading `/' from member names
2026-07-01T17:17:54.431573038Z tar: Removing leading `/' from member names
2026-07-01T17:17:54.446644098Z tar: Removing leading `/' from member names
2026-07-01T17:17:54.451287780Z tar: Removing leading `/' from member names
2026-07-01T17:17:54.457515478Z Wrote inspect data to must-gather.
2026-07-01T17:17:54.469980450Z tar: Removing leading `/' from member names
2026-07-01T17:17:54.483907264Z tar: Removing leading `/' from member names
2026-07-01T17:17:54.514766296Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:54.535956651Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-01T17:17:54.581044045Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-01T17:17:54.700345560Z Wrote inspect data to must-gather.
2026-07-01T17:17:54.757967069Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:54.761857318Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-01T17:17:54.802560597Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-01T17:17:54.951187249Z Wrote inspect data to must-gather.
2026-07-01T17:17:54.979601260Z INFO: Waiting for network log collection to complete ...
2026-07-01T17:17:54.979716725Z INFO: Waiting for ovnk database copies to complete ...
2026-07-01T17:17:54.980261107Z INFO: Copying ovnk databases complete.
2026-07-01T17:17:54.981974539Z 2.6M	must-gather/network_logs/ovnk_database_store
2026-07-01T17:17:54.984258831Z ovnk_database_store/
2026-07-01T17:17:54.984308620Z ovnk_database_store/ovnkube-node-j4tb4_nbdb
2026-07-01T17:17:54.989906994Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:54.993501069Z ovnk_database_store/ovnkube-node-snn2l_nbdb
2026-07-01T17:17:55.001098732Z ovnk_database_store/ovnkube-node-snn2l_sbdb
2026-07-01T17:17:55.017117485Z ovnk_database_store/ovnkube-node-j4tb4_sbdb
2026-07-01T17:17:55.032302356Z ovnk_database_store/ovnkube-node-mvdwc_nbdb
2026-07-01T17:17:55.038679846Z ovnk_database_store/ovnkube-node-mvdwc_sbdb
2026-07-01T17:17:55.060891244Z INFO: Network log collection complete.
2026-07-01T17:17:55.165817803Z W0701 17:17:55.165764    1542 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-01T17:17:55.170439673Z Wrote inspect data to must-gather.
2026-07-01T17:17:55.205255192Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:55.208038281Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-01T17:17:55.228423621Z INFO: Getting status from alertmanager-main-0
2026-07-01T17:17:55.283632320Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-01T17:17:55.460296883Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:55.461964595Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-01T17:17:55.464556002Z INFO: Fallback to identify the container image from release info
2026-07-01T17:17:55.481816540Z Gathering data for ns/openshift-network-console...
2026-07-01T17:17:55.656572865Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:55.659030250Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-01T17:17:55.824690130Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:55.937719777Z Gathering data for ns/openshift-console-operator...
2026-07-01T17:17:56.020287898Z Wrote inspect data to /must-gather/istio.
2026-07-01T17:17:56.329317622Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2c4f9949a07a5338dd10a5abcb8d6d3f5eebe93333139869f9eec8802800162f
2026-07-01T17:17:56.360867792Z Gathering data for ns/openshift-console...
2026-07-01T17:17:56.464894262Z daemonset.apps/perf-node-gather-daemonset created
2026-07-01T17:17:56.598629668Z Waiting for performance profile collector pods to become ready: 1
2026-07-01T17:17:56.685661756Z Done executing Istio gather script
2026-07-01T17:17:56.905772481Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-01T17:17:57.409768280Z Gathering data for ns/openshift-dns-operator...
2026-07-01T17:17:57.623209595Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-01T17:17:57.718838201Z Waiting for performance profile collector pods to become ready: 2
2026-07-01T17:17:57.722446376Z Gathering data for ns/openshift-dns...
2026-07-01T17:17:58.437903037Z Gathering data for ns/openshift-image-registry...
2026-07-01T17:17:58.848344544Z Waiting for performance profile collector pods to become ready: 3
2026-07-01T17:17:58.931896455Z Gathering data for ns/openshift-ingress-operator...
2026-07-01T17:17:59.269942427Z Gathering data for ns/openshift-ingress...
2026-07-01T17:17:59.614104676Z Gathering data for ns/openshift-ingress-canary...
2026-07-01T17:17:59.969127757Z Waiting for performance profile collector pods to become ready: 4
2026-07-01T17:18:00.109501119Z Gathering data for ns/openshift-insights...
2026-07-01T17:18:01.081322325Z Waiting for performance profile collector pods to become ready: 5
2026-07-01T17:18:02.139720887Z Gathering data for ns/openshift-config...
2026-07-01T17:18:02.211683301Z Waiting for performance profile collector pods to become ready: 6
2026-07-01T17:18:02.512045916Z Gathering data for ns/openshift-config-managed...
2026-07-01T17:18:02.628328695Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-01T17:18:02.967086144Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-01T17:18:03.307769012Z Gathering data for ns/openshift-kube-apiserver...
2026-07-01T17:18:03.339863157Z Waiting for performance profile collector pods to become ready: 7
2026-07-01T17:18:03.634904087Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-01T17:18:03.941938249Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-01T17:18:04.276312780Z Gathering data for ns/openshift-kube-scheduler...
2026-07-01T17:18:04.451732184Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-01T17:18:04.558600739Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-01T17:18:04.706932577Z Collecting performance related data for node ip-10-0-139-65.ec2.internal
2026-07-01T17:18:04.716595527Z Collecting performance related data for node ip-10-0-137-171.ec2.internal
2026-07-01T17:18:04.727273884Z Collecting performance related data for node ip-10-0-131-254.ec2.internal
2026-07-01T17:18:04.912662345Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-01T17:18:05.297258326Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-01T17:18:05.685101031Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-01T17:18:06.368151380Z Gathering data for ns/openshift-multus...
2026-07-01T17:18:07.634211642Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-01T17:18:07.931911838Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-01T17:18:09.284034218Z Gathering data for ns/openshift-host-network...
2026-07-01T17:18:09.630173061Z Gathering data for ns/openshift-network-diagnostics...
2026-07-01T17:18:10.254062831Z Gathering data for ns/openshift-network-node-identity...
2026-07-01T17:18:10.703390947Z Gathering data for ns/openshift-network-operator...
2026-07-01T17:18:11.106146257Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-01T17:18:11.382787717Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-01T17:18:11.929865497Z Gathering data for ns/openshift-apiserver-operator...
2026-07-01T17:18:12.233567317Z Gathering data for ns/openshift-apiserver...
2026-07-01T17:18:12.640540490Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-01T17:18:12.682489857Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-01T17:18:12.682727129Z Collecting kubelet logs for node ip-10-0-139-65.ec2.internal
2026-07-01T17:18:12.684873577Z Collecting kubelet logs for node ip-10-0-137-171.ec2.internal
2026-07-01T17:18:12.686846681Z Collecting kubelet logs for node ip-10-0-131-254.ec2.internal
2026-07-01T17:18:13.038478281Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-dslx9 namespace
2026-07-01T17:18:13.044583184Z INFO: Node performance data collection complete.
2026-07-01T17:18:13.049264618Z Gathering data for ns/openshift-controller-manager...
2026-07-01T17:18:13.410458286Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-01T17:18:14.064631828Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-01T17:18:14.360425175Z Gathering data for ns/openshift-service-ca-operator...
2026-07-01T17:18:14.672546862Z Gathering data for ns/openshift-service-ca...
2026-07-01T17:18:15.073402182Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-01T17:18:15.977419747Z Wrote inspect data to must-gather.
2026-07-01T17:18:15.977466259Z error: inspection completed with the errors occurred while gathering data:
2026-07-01T17:18:15.977466259Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-01T17:18:16.102909016Z error: the server doesn't have a resource type "clusters"
2026-07-01T17:18:16.230241419Z Caches written to disk