2026-06-08T22:17:15.843424692Z [disk usage checker] Started
2026-06-08T22:17:15.846970281Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-08T22:17:16.028557698Z Gathering data for ns/openshift-cluster-version...
2026-06-08T22:17:16.169526684Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T22:17:16.262982619Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T22:17:16.344593416Z Gathering data for ns/default...
2026-06-08T22:17:16.629476966Z Gathering data for ns/openshift...
2026-06-08T22:17:17.099324587Z Gathering data for ns/kube-system...
2026-06-08T22:17:17.889685634Z Gathering data for ns/openshift-etcd...
2026-06-08T22:17:18.219499924Z Wrote inspect data to must-gather.
2026-06-08T22:17:18.219533206Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T22:17:18.219533206Z     namespaces "assisted-installer" not found
2026-06-08T22:17:18.375658273Z Waiting on subprocesses to finish execution.
2026-06-08T22:17:18.381268263Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-08T22:17:18.382916400Z INFO: Gathering HAProxy config files
2026-06-08T22:17:18.387331298Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-08T22:17:18.387609235Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-08T22:17:18.389350144Z INFO: Collecting host service logs for crio
2026-06-08T22:17:18.389684462Z INFO: Collecting host service logs for kubelet
2026-06-08T22:17:18.390054284Z INFO: Collecting host service logs for rpm-ostreed
2026-06-08T22:17:18.390484765Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-08T22:17:18.390883658Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-08T22:17:18.391224370Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-08T22:17:18.391547912Z INFO: Collecting host service logs for NetworkManager
2026-06-08T22:17:18.391936807Z INFO: Collecting host service logs for openvswitch
2026-06-08T22:17:18.392299593Z INFO: Collecting host service logs for ovs-configuration
2026-06-08T22:17:18.392663337Z INFO: Collecting host service logs for ovsdb-server
2026-06-08T22:17:18.392729266Z Executing Istio gather script
2026-06-08T22:17:18.393037084Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-08T22:17:18.393422264Z INFO: Waiting for worker host service log collection to complete ...
2026-06-08T22:17:18.786504725Z INFO: Waiting for node performance related collection to complete ...
2026-06-08T22:17:18.863837990Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-08T22:17:19.084607205Z Wrote inspect data to must-gather.
2026-06-08T22:17:19.223666661Z No resources found in openshift-etcd namespace.
2026-06-08T22:17:19.358196471Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-08T22:17:19.361690063Z INFO: Collecting Insights Archives from insights-operator-896549976-c5gq6
2026-06-08T22:17:19.361728609Z insights-runtime-extractor-4bstq
2026-06-08T22:17:19.361746270Z insights-runtime-extractor-j85hp
2026-06-08T22:17:19.361763813Z insights-runtime-extractor-xkhl7
2026-06-08T22:17:19.457117247Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-08T22:17:19.466622525Z error: the server doesn't have a resource type "performanceprofile"
2026-06-08T22:17:19.498619801Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-08T22:17:19.619821907Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-08T22:17:19.741464722Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-08T22:17:19.741464722Z See 'oc rsync -h' for help and examples
2026-06-08T22:17:19.813368500Z INFO: "metallb-operator" not detected. Skipping.
2026-06-08T22:17:19.826038460Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-08T22:17:19.846083153Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-08T22:17:19.846167022Z INFO: on-disk MachineConfig config collection complete.
2026-06-08T22:17:19.873068182Z INFO: Worker host service log collection to complete.
2026-06-08T22:17:19.989604882Z No resources found
2026-06-08T22:17:20.253649621Z Wrote inspect data to must-gather.
2026-06-08T22:17:20.398745362Z tar: Removing leading `/' from member names
2026-06-08T22:17:20.405221493Z INFO: HAProxy config collection complete.
2026-06-08T22:17:20.561621775Z Wrote inspect data to must-gather.
2026-06-08T22:17:20.628434299Z Wrote inspect data to must-gather.
2026-06-08T22:17:20.657598869Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T22:17:20.658934886Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-08T22:17:20.671962706Z ERROR: No running kube-apiserver pods found
2026-06-08T22:17:20.700543086Z Gathering data for ns/openshift-monitoring...
2026-06-08T22:17:20.724254471Z Wrote inspect data to must-gather.
2026-06-08T22:17:20.785233868Z No resources found
2026-06-08T22:17:20.836687757Z INFO: 'previous-logs' folder not found on ip-10-0-129-67.ec2.internal, skipping...
2026-06-08T22:17:20.872673260Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-08T22:17:20.896523902Z error: the server doesn't have a resource type "machineconfigs"
2026-06-08T22:17:21.038411257Z Wrote inspect data to must-gather.
2026-06-08T22:17:21.140672759Z Wrote inspect data to must-gather.
2026-06-08T22:17:21.173732511Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-08T22:17:21.237370858Z INFO: 'previous-logs' folder not found on ip-10-0-132-240.ec2.internal, skipping...
2026-06-08T22:17:21.256964797Z INFO: Getting rules from prometheus-k8s-0
2026-06-08T22:17:21.396491304Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-08T22:17:21.438283457Z Wrote inspect data to must-gather.
2026-06-08T22:17:21.456961916Z Wrote inspect data to must-gather.
2026-06-08T22:17:21.518096470Z INFO: 'previous-logs' folder not found on ip-10-0-138-88.ec2.internal, skipping...
2026-06-08T22:17:21.518132206Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-08T22:17:21.518132206Z INFO: Machine Config Daemon termination log collection complete.
2026-06-08T22:17:21.619979546Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-08T22:17:21.682160614Z INFO: Getting status/config from prometheus-k8s-0
2026-06-08T22:17:21.687125742Z Wrote inspect data to must-gather.
2026-06-08T22:17:21.864124224Z Wrote inspect data to must-gather.
2026-06-08T22:17:21.920610473Z Wrote inspect data to must-gather.
2026-06-08T22:17:22.075553595Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-08T22:17:22.088857072Z Wrote inspect data to must-gather.
2026-06-08T22:17:22.193602897Z INFO: INTERCONNECT MODE
2026-06-08T22:17:22.193678190Z INFO: Gathering ovn-kubernetes DBs
2026-06-08T22:17:22.303758176Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-08T22:17:22.326288007Z Wrote inspect data to must-gather.
2026-06-08T22:17:22.348446749Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-08T22:17:22.363236410Z INFO: Gathering OVN_Northbound from ovnkube-node-4dxnf...
2026-06-08T22:17:22.363531666Z INFO: Gathering OVN_Northbound from ovnkube-node-qqzs7...
2026-06-08T22:17:22.363801341Z INFO: Gathering OVN_Northbound from ovnkube-node-wqrdx...
2026-06-08T22:17:22.364025501Z INFO: Gathering OVN_Southbound from ovnkube-node-4dxnf...
2026-06-08T22:17:22.364288026Z INFO: Gathering OVN_Southbound from ovnkube-node-qqzs7...
2026-06-08T22:17:22.364569967Z INFO: Gathering OVN_Southbound from ovnkube-node-wqrdx...
2026-06-08T22:17:22.398510763Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-08T22:17:22.636699560Z tar: Removing leading `/' from member names
2026-06-08T22:17:22.666858741Z tar: Removing leading `/' from member names
2026-06-08T22:17:22.671269359Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-08T22:17:22.684961672Z Wrote inspect data to must-gather.
2026-06-08T22:17:22.685591810Z tar: Removing leading `/' from member names
2026-06-08T22:17:22.699316051Z tar: Removing leading `/' from member names
2026-06-08T22:17:22.722503920Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:22.739119787Z tar: Removing leading `/' from member names
2026-06-08T22:17:22.742470639Z tar: Removing leading `/' from member names
2026-06-08T22:17:22.924296873Z Wrote inspect data to must-gather.
2026-06-08T22:17:22.946890593Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-08T22:17:22.978575747Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:22.986709322Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-08T22:17:23.167382172Z W0608 22:17:23.167337    1478 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-08T22:17:23.173573563Z Wrote inspect data to must-gather.
2026-06-08T22:17:23.188710333Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-08T22:17:23.220616062Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:23.235415923Z INFO: Waiting for network log collection to complete ...
2026-06-08T22:17:23.235495129Z INFO: Waiting for ovnk database copies to complete ...
2026-06-08T22:17:23.236023298Z INFO: Copying ovnk databases complete.
2026-06-08T22:17:23.237962234Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-06-08T22:17:23.241270711Z ovnk_database_store/
2026-06-08T22:17:23.241323908Z ovnk_database_store/ovnkube-node-qqzs7_nbdb
2026-06-08T22:17:23.254198867Z ovnk_database_store/ovnkube-node-4dxnf_sbdb
2026-06-08T22:17:23.273342539Z ovnk_database_store/ovnkube-node-qqzs7_sbdb
2026-06-08T22:17:23.292510882Z ovnk_database_store/ovnkube-node-wqrdx_nbdb
2026-06-08T22:17:23.299662692Z ovnk_database_store/ovnkube-node-wqrdx_sbdb
2026-06-08T22:17:23.317221539Z ovnk_database_store/ovnkube-node-4dxnf_nbdb
2026-06-08T22:17:23.329332365Z INFO: Network log collection complete.
2026-06-08T22:17:23.373418945Z INFO: Getting status from alertmanager-main-0
2026-06-08T22:17:23.400890316Z Gathering data for ns/openshift-network-console...
2026-06-08T22:17:23.465259645Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:23.469672664Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-08T22:17:23.528461556Z Wrote inspect data to must-gather.
2026-06-08T22:17:23.676399606Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:23.784049066Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-08T22:17:23.786963743Z INFO: Fallback to identify the container image from release info
2026-06-08T22:17:23.837074744Z Gathering data for ns/openshift-console-operator...
2026-06-08T22:17:23.885467198Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:23.888177862Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-08T22:17:24.101234173Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:24.221573645Z Gathering data for ns/openshift-console...
2026-06-08T22:17:24.281544948Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:24.283707633Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-08T22:17:24.430661288Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-08T22:17:24.447474689Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:24.546786865Z daemonset.apps/perf-node-gather-daemonset created
2026-06-08T22:17:24.654763822Z Wrote inspect data to /must-gather/istio.
2026-06-08T22:17:24.695197044Z Waiting for performance profile collector pods to become ready: 1
2026-06-08T22:17:24.771208434Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-08T22:17:25.305275091Z Gathering data for ns/openshift-dns-operator...
2026-06-08T22:17:25.356413112Z Done executing Istio gather script
2026-06-08T22:17:25.568136859Z Gathering data for ns/openshift-dns...
2026-06-08T22:17:25.805152234Z Waiting for performance profile collector pods to become ready: 2
2026-06-08T22:17:25.947671506Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-08T22:17:26.145385795Z Gathering data for ns/openshift-image-registry...
2026-06-08T22:17:26.639602745Z Gathering data for ns/openshift-ingress-operator...
2026-06-08T22:17:26.921377803Z Waiting for performance profile collector pods to become ready: 3
2026-06-08T22:17:26.997336915Z Gathering data for ns/openshift-ingress...
2026-06-08T22:17:27.307760993Z Gathering data for ns/openshift-ingress-canary...
2026-06-08T22:17:27.689025900Z Gathering data for ns/openshift-insights...
2026-06-08T22:17:28.034357738Z Waiting for performance profile collector pods to become ready: 4
2026-06-08T22:17:29.140705278Z Waiting for performance profile collector pods to become ready: 5
2026-06-08T22:17:29.487881402Z Gathering data for ns/openshift-jobset-operator...
2026-06-08T22:17:30.249956300Z Waiting for performance profile collector pods to become ready: 6
2026-06-08T22:17:30.250204145Z Gathering data for ns/openshift-config...
2026-06-08T22:17:30.577031949Z Gathering data for ns/openshift-config-managed...
2026-06-08T22:17:30.952480257Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-08T22:17:30.981925568Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-08T22:17:31.286472456Z Gathering data for ns/openshift-kube-apiserver...
2026-06-08T22:17:31.374014982Z Waiting for performance profile collector pods to become ready: 7
2026-06-08T22:17:31.552343176Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-08T22:17:31.814576336Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-08T22:17:32.081870584Z Gathering data for ns/openshift-kube-scheduler...
2026-06-08T22:17:32.365352960Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-08T22:17:32.490342731Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-08T22:17:32.660400282Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-08T22:17:32.728981285Z Collecting performance related data for node ip-10-0-129-67.ec2.internal
2026-06-08T22:17:32.736069441Z Collecting performance related data for node ip-10-0-138-88.ec2.internal
2026-06-08T22:17:32.743160444Z Collecting performance related data for node ip-10-0-132-240.ec2.internal
2026-06-08T22:17:33.083842098Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-08T22:17:33.479924241Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-08T22:17:34.077521672Z Gathering data for ns/openshift-multus...
2026-06-08T22:17:35.609500981Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-08T22:17:35.957331117Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-08T22:17:36.789665797Z Gathering data for ns/openshift-host-network...
2026-06-08T22:17:37.089365807Z Gathering data for ns/openshift-network-diagnostics...
2026-06-08T22:17:37.623948043Z Gathering data for ns/openshift-network-node-identity...
2026-06-08T22:17:38.038261119Z Gathering data for ns/openshift-network-operator...
2026-06-08T22:17:38.434028446Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-08T22:17:38.682600075Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-08T22:17:39.199046678Z Gathering data for ns/openshift-apiserver-operator...
2026-06-08T22:17:39.521357527Z Gathering data for ns/openshift-apiserver...
2026-06-08T22:17:39.898003344Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-08T22:17:40.155501106Z Gathering data for ns/openshift-controller-manager...
2026-06-08T22:17:40.412928541Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-08T22:17:40.636008191Z Collecting kubelet logs for node ip-10-0-129-67.ec2.internal
2026-06-08T22:17:40.637732259Z Collecting kubelet logs for node ip-10-0-138-88.ec2.internal
2026-06-08T22:17:40.639964004Z Collecting kubelet logs for node ip-10-0-132-240.ec2.internal
2026-06-08T22:17:40.963600951Z [disk usage checker] Volume usage percentage: current = 16 ; allowed = 70
2026-06-08T22:17:40.971743979Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-jh4bb namespace
2026-06-08T22:17:40.976228792Z INFO: Node performance data collection complete.
2026-06-08T22:17:41.110929225Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-08T22:17:41.408896020Z Gathering data for ns/openshift-service-ca-operator...
2026-06-08T22:17:41.747391288Z Gathering data for ns/openshift-service-ca...
2026-06-08T22:17:42.109739058Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-08T22:17:42.935384928Z Wrote inspect data to must-gather.
2026-06-08T22:17:42.935423815Z error: inspection completed with the errors occurred while gathering data:
2026-06-08T22:17:42.935423815Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-08T22:17:43.053711060Z error: the server doesn't have a resource type "clusters"
2026-06-08T22:17:43.184009659Z Caches written to disk