2026-07-23T18:19:15.210785499Z [disk usage checker] Started
2026-07-23T18:19:15.213873134Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-23T18:19:15.392911834Z Gathering data for ns/openshift-cluster-version...
2026-07-23T18:19:15.509819238Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-23T18:19:15.585948134Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-23T18:19:15.658783874Z Gathering data for ns/default...
2026-07-23T18:19:15.960581599Z Gathering data for ns/openshift...
2026-07-23T18:19:16.337072215Z Gathering data for ns/kube-system...
2026-07-23T18:19:16.961283744Z Gathering data for ns/openshift-etcd...
2026-07-23T18:19:17.231242964Z Wrote inspect data to must-gather.
2026-07-23T18:19:17.231276359Z error: inspection completed with the errors occurred while gathering data:
2026-07-23T18:19:17.231276359Z     namespaces "assisted-installer" not found
2026-07-23T18:19:17.621489610Z Waiting on subprocesses to finish execution.
2026-07-23T18:19:17.636652728Z Executing Istio gather script
2026-07-23T18:19:17.637392692Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-23T18:19:17.642104799Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-23T18:19:17.646749695Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-23T18:19:17.651367398Z INFO: Waiting for node performance related collection to complete ...
2026-07-23T18:19:17.670299259Z INFO: Collecting host service logs for crio
2026-07-23T18:19:17.670596406Z INFO: Collecting host service logs for kubelet
2026-07-23T18:19:17.670904789Z INFO: Collecting host service logs for rpm-ostreed
2026-07-23T18:19:17.671234221Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-23T18:19:17.671587655Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-23T18:19:17.671917430Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-23T18:19:17.672245307Z INFO: Collecting host service logs for NetworkManager
2026-07-23T18:19:17.672600176Z INFO: Collecting host service logs for openvswitch
2026-07-23T18:19:17.672951920Z INFO: Collecting host service logs for ovs-configuration
2026-07-23T18:19:17.673286225Z INFO: Collecting host service logs for ovsdb-server
2026-07-23T18:19:17.673671127Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-23T18:19:17.674075621Z INFO: Waiting for worker host service log collection to complete ...
2026-07-23T18:19:17.701556496Z INFO: Gathering HAProxy config files
2026-07-23T18:19:18.043313214Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-x2vlf
2026-07-23T18:19:18.043362787Z insights-runtime-extractor-ctjdh
2026-07-23T18:19:18.043380662Z insights-runtime-extractor-vz4td
2026-07-23T18:19:18.043398842Z insights-runtime-extractor-xd9x5
2026-07-23T18:19:18.048920466Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-23T18:19:18.107525547Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-23T18:19:18.250807300Z INFO: "metallb-operator" not detected. Skipping.
2026-07-23T18:19:18.462868212Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-23T18:19:18.462868212Z See 'oc rsync -h' for help and examples
2026-07-23T18:19:18.551800419Z error: the server doesn't have a resource type "performanceprofile"
2026-07-23T18:19:18.582905614Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-23T18:19:18.764938536Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-23T18:19:18.818844478Z Wrote inspect data to must-gather.
2026-07-23T18:19:18.839262748Z No resources found
2026-07-23T18:19:19.037461043Z ERROR: No running kube-apiserver pods found
2026-07-23T18:19:19.509094385Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-23T18:19:19.521305421Z No resources found in openshift-etcd namespace.
2026-07-23T18:19:19.533566043Z Wrote inspect data to must-gather.
2026-07-23T18:19:19.556172161Z INFO: Worker host service log collection to complete.
2026-07-23T18:19:19.578616789Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-23T18:19:19.578698125Z INFO: on-disk MachineConfig config collection complete.
2026-07-23T18:19:19.596937412Z Gathering data for ns/openshift-monitoring...
2026-07-23T18:19:19.603728387Z INFO: 'previous-logs' folder not found on ip-10-0-128-242.ec2.internal, skipping...
2026-07-23T18:19:19.624190050Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-23T18:19:19.686751567Z Wrote inspect data to must-gather.
2026-07-23T18:19:19.789230908Z Wrote inspect data to must-gather.
2026-07-23T18:19:19.912397337Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-23T18:19:19.971744032Z Wrote inspect data to must-gather.
2026-07-23T18:19:19.993144380Z Wrote inspect data to must-gather.
2026-07-23T18:19:20.005997311Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-23T18:19:20.100051865Z No resources found
2026-07-23T18:19:20.219992479Z Wrote inspect data to must-gather.
2026-07-23T18:19:20.226879733Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-23T18:19:20.278756537Z error: the server doesn't have a resource type "machineconfigs"
2026-07-23T18:19:20.293251418Z Wrote inspect data to must-gather.
2026-07-23T18:19:20.301474120Z INFO: 'previous-logs' folder not found on ip-10-0-129-157.ec2.internal, skipping...
2026-07-23T18:19:20.302134874Z tar: Removing leading `/' from member names
2026-07-23T18:19:20.307550802Z INFO: HAProxy config collection complete.
2026-07-23T18:19:20.375173160Z INFO: Getting rules from prometheus-k8s-0
2026-07-23T18:19:20.484077554Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-23T18:19:20.600289595Z INFO: 'previous-logs' folder not found on ip-10-0-135-200.ec2.internal, skipping...
2026-07-23T18:19:20.600317727Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-23T18:19:20.600334194Z INFO: Machine Config Daemon termination log collection complete.
2026-07-23T18:19:20.609009217Z Wrote inspect data to must-gather.
2026-07-23T18:19:20.703363977Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-23T18:19:20.721706236Z Wrote inspect data to must-gather.
2026-07-23T18:19:20.780321081Z INFO: Getting status/config from prometheus-k8s-0
2026-07-23T18:19:20.856364105Z Wrote inspect data to must-gather.
2026-07-23T18:19:20.878827358Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-23T18:19:21.099003790Z Wrote inspect data to must-gather.
2026-07-23T18:19:21.115887924Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-23T18:19:21.132782868Z INFO: INTERCONNECT MODE
2026-07-23T18:19:21.132822171Z INFO: Gathering ovn-kubernetes DBs
2026-07-23T18:19:21.181954048Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-23T18:19:21.305271231Z INFO: Gathering OVN_Northbound from ovnkube-node-9s8c5...
2026-07-23T18:19:21.305550935Z INFO: Gathering OVN_Northbound from ovnkube-node-mxrwf...
2026-07-23T18:19:21.305807790Z INFO: Gathering OVN_Northbound from ovnkube-node-s2l8h...
2026-07-23T18:19:21.306062786Z INFO: Gathering OVN_Southbound from ovnkube-node-9s8c5...
2026-07-23T18:19:21.306316337Z INFO: Gathering OVN_Southbound from ovnkube-node-mxrwf...
2026-07-23T18:19:21.306596252Z INFO: Gathering OVN_Southbound from ovnkube-node-s2l8h...
2026-07-23T18:19:21.374080746Z Wrote inspect data to must-gather.
2026-07-23T18:19:21.431137700Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-23T18:19:21.497641780Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-23T18:19:21.601305661Z tar: Removing leading `/' from member names
2026-07-23T18:19:21.609643142Z tar: Removing leading `/' from member names
2026-07-23T18:19:21.635996062Z tar: Removing leading `/' from member names
2026-07-23T18:19:21.652899033Z tar: Removing leading `/' from member names
2026-07-23T18:19:21.664814334Z tar: Removing leading `/' from member names
2026-07-23T18:19:21.685197247Z tar: Removing leading `/' from member names
2026-07-23T18:19:21.733903658Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:21.740249297Z W0723 18:19:21.740206    1328 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-23T18:19:21.747348599Z Wrote inspect data to must-gather.
2026-07-23T18:19:21.811970368Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-23T18:19:21.888711269Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-23T18:19:21.995116764Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:21.998257038Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-23T18:19:22.084456182Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-23T18:19:22.096096513Z Wrote inspect data to must-gather.
2026-07-23T18:19:22.116013746Z Gathering data for ns/openshift-network-console...
2026-07-23T18:19:22.144356129Z INFO: Waiting for network log collection to complete ...
2026-07-23T18:19:22.144410022Z INFO: Waiting for ovnk database copies to complete ...
2026-07-23T18:19:22.144908805Z INFO: Copying ovnk databases complete.
2026-07-23T18:19:22.146546740Z 2.7M	must-gather/network_logs/ovnk_database_store
2026-07-23T18:19:22.148826677Z ovnk_database_store/
2026-07-23T18:19:22.148876345Z ovnk_database_store/ovnkube-node-s2l8h_nbdb
2026-07-23T18:19:22.162747384Z ovnk_database_store/ovnkube-node-mxrwf_nbdb
2026-07-23T18:19:22.170768824Z ovnk_database_store/ovnkube-node-mxrwf_sbdb
2026-07-23T18:19:22.190371329Z ovnk_database_store/ovnkube-node-s2l8h_sbdb
2026-07-23T18:19:22.209045700Z ovnk_database_store/ovnkube-node-9s8c5_sbdb
2026-07-23T18:19:22.219450938Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:22.227518443Z ovnk_database_store/ovnkube-node-9s8c5_nbdb
2026-07-23T18:19:22.238236148Z INFO: Network log collection complete.
2026-07-23T18:19:22.414168010Z Wrote inspect data to must-gather.
2026-07-23T18:19:22.422898473Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:22.425946802Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-23T18:19:22.479038895Z INFO: Getting status from alertmanager-main-0
2026-07-23T18:19:22.611854665Z Gathering data for ns/openshift-console-operator...
2026-07-23T18:19:22.644513181Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:22.740943933Z Wrote inspect data to must-gather.
2026-07-23T18:19:22.854678510Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:22.858056830Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-23T18:19:22.995214128Z Gathering data for ns/openshift-console...
2026-07-23T18:19:23.031919488Z Wrote inspect data to must-gather.
2026-07-23T18:19:23.065250390Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:23.256857803Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:23.259340454Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-23T18:19:23.291748100Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-23T18:19:23.294021222Z INFO: Fallback to identify the container image from release info
2026-07-23T18:19:23.431312984Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:23.491819718Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-23T18:19:23.612764417Z Wrote inspect data to /must-gather/istio.
2026-07-23T18:19:23.949865048Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-23T18:19:23.984753198Z Gathering data for ns/openshift-dns-operator...
2026-07-23T18:19:24.081215526Z daemonset.apps/perf-node-gather-daemonset created
2026-07-23T18:19:24.200582397Z Waiting for performance profile collector pods to become ready: 1
2026-07-23T18:19:24.291479213Z Done executing Istio gather script
2026-07-23T18:19:24.291886753Z Gathering data for ns/openshift-dns...
2026-07-23T18:19:24.838166400Z Gathering data for ns/openshift-image-registry...
2026-07-23T18:19:25.232433921Z Gathering data for ns/openshift-ingress-operator...
2026-07-23T18:19:25.234324887Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-23T18:19:25.325421490Z Waiting for performance profile collector pods to become ready: 2
2026-07-23T18:19:25.525694281Z Gathering data for ns/openshift-ingress...
2026-07-23T18:19:25.808518558Z Gathering data for ns/openshift-ingress-canary...
2026-07-23T18:19:26.129790360Z Gathering data for ns/openshift-insights...
2026-07-23T18:19:26.440731988Z Waiting for performance profile collector pods to become ready: 3
2026-07-23T18:19:27.548379563Z Waiting for performance profile collector pods to become ready: 4
2026-07-23T18:19:27.860669773Z Gathering data for ns/openshift-jobset-operator...
2026-07-23T18:19:28.596442116Z Gathering data for ns/openshift-config...
2026-07-23T18:19:28.666134239Z Waiting for performance profile collector pods to become ready: 5
2026-07-23T18:19:28.862201559Z Gathering data for ns/openshift-config-managed...
2026-07-23T18:19:29.230788761Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-23T18:19:29.450411895Z Gathering data for ns/openshift-kube-apiserver...
2026-07-23T18:19:29.667517641Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-23T18:19:29.782177831Z Waiting for performance profile collector pods to become ready: 6
2026-07-23T18:19:29.952848601Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-23T18:19:30.197419446Z Gathering data for ns/openshift-kube-scheduler...
2026-07-23T18:19:30.239411908Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-23T18:19:30.430421897Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-23T18:19:30.675392116Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-23T18:19:30.890089221Z Waiting for performance profile collector pods to become ready: 7
2026-07-23T18:19:30.955764605Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-23T18:19:31.304525919Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-23T18:19:31.802456428Z Gathering data for ns/openshift-multus...
2026-07-23T18:19:31.986856608Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-23T18:19:32.188914354Z Collecting performance related data for node ip-10-0-135-200.ec2.internal
2026-07-23T18:19:32.195485951Z Collecting performance related data for node ip-10-0-128-242.ec2.internal
2026-07-23T18:19:32.202560831Z Collecting performance related data for node ip-10-0-129-157.ec2.internal
2026-07-23T18:19:33.147187943Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-23T18:19:34.241047544Z Gathering data for ns/openshift-host-network...
2026-07-23T18:19:34.595916406Z Gathering data for ns/openshift-network-diagnostics...
2026-07-23T18:19:35.099622111Z Gathering data for ns/openshift-network-node-identity...
2026-07-23T18:19:35.247299250Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-23T18:19:35.450390019Z Gathering data for ns/openshift-network-operator...
2026-07-23T18:19:35.773917725Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-23T18:19:36.002046554Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-23T18:19:36.428471799Z Gathering data for ns/openshift-apiserver-operator...
2026-07-23T18:19:36.700281110Z Gathering data for ns/openshift-apiserver...
2026-07-23T18:19:37.057138999Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-23T18:19:37.284558498Z Gathering data for ns/openshift-controller-manager...
2026-07-23T18:19:37.523297978Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-23T18:19:38.104575887Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-23T18:19:38.396647960Z Gathering data for ns/openshift-service-ca-operator...
2026-07-23T18:19:38.676181587Z Gathering data for ns/openshift-service-ca...
2026-07-23T18:19:38.992725265Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-23T18:19:39.844967687Z Wrote inspect data to must-gather.
2026-07-23T18:19:39.845009375Z error: inspection completed with the errors occurred while gathering data:
2026-07-23T18:19:39.845009375Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-23T18:19:40.061354438Z Collecting kubelet logs for node ip-10-0-135-200.ec2.internal
2026-07-23T18:19:40.062988775Z Collecting kubelet logs for node ip-10-0-128-242.ec2.internal
2026-07-23T18:19:40.064747099Z Collecting kubelet logs for node ip-10-0-129-157.ec2.internal
2026-07-23T18:19:40.251794596Z [disk usage checker] Volume usage percentage: current = 11 ; allowed = 70
2026-07-23T18:19:40.361857072Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-v2h2z namespace
2026-07-23T18:19:40.366484593Z INFO: Node performance data collection complete.
2026-07-23T18:19:40.483026223Z error: the server doesn't have a resource type "clusters"
2026-07-23T18:19:40.649149183Z Caches written to disk