2026-07-13T20:09:25.430963069Z [disk usage checker] Started
2026-07-13T20:09:25.434509867Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T20:09:25.632576634Z Gathering data for ns/openshift-cluster-version...
2026-07-13T20:09:25.771975462Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-13T20:09:25.877234619Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T20:09:25.962777747Z Gathering data for ns/default...
2026-07-13T20:09:26.243746559Z Gathering data for ns/openshift...
2026-07-13T20:09:26.632335973Z Gathering data for ns/kube-system...
2026-07-13T20:09:27.322442761Z Gathering data for ns/openshift-etcd...
2026-07-13T20:09:27.645797166Z Wrote inspect data to must-gather.
2026-07-13T20:09:27.645846076Z error: inspection completed with the errors occurred while gathering data:
2026-07-13T20:09:27.645846076Z     namespaces "assisted-installer" not found
2026-07-13T20:09:27.986004168Z Waiting on subprocesses to finish execution.
2026-07-13T20:09:27.993590844Z Executing Istio gather script
2026-07-13T20:09:28.000397423Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-13T20:09:28.006687707Z INFO: Waiting for node performance related collection to complete ...
2026-07-13T20:09:28.006849219Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-13T20:09:28.009151715Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-13T20:09:28.009917696Z INFO: Gathering HAProxy config files
2026-07-13T20:09:28.018323634Z INFO: Collecting host service logs for crio
2026-07-13T20:09:28.018675777Z INFO: Collecting host service logs for kubelet
2026-07-13T20:09:28.019040769Z INFO: Collecting host service logs for rpm-ostreed
2026-07-13T20:09:28.019356629Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-13T20:09:28.019689637Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-13T20:09:28.020021787Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-13T20:09:28.020358181Z INFO: Collecting host service logs for NetworkManager
2026-07-13T20:09:28.020680393Z INFO: Collecting host service logs for openvswitch
2026-07-13T20:09:28.021017383Z INFO: Collecting host service logs for ovs-configuration
2026-07-13T20:09:28.021362753Z INFO: Collecting host service logs for ovsdb-server
2026-07-13T20:09:28.021698572Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-13T20:09:28.022112598Z INFO: Waiting for worker host service log collection to complete ...
2026-07-13T20:09:28.669125904Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-13T20:09:28.733583835Z No resources found
2026-07-13T20:09:28.828562703Z No resources found in openshift-etcd namespace.
2026-07-13T20:09:28.853627243Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-13T20:09:28.898776017Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-13T20:09:28.898896205Z INFO: on-disk MachineConfig config collection complete.
2026-07-13T20:09:29.117088310Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-13T20:09:29.230000159Z INFO: "metallb-operator" not detected. Skipping.
2026-07-13T20:09:29.263234399Z Wrote inspect data to must-gather.
2026-07-13T20:09:29.390047885Z error: the server doesn't have a resource type "performanceprofile"
2026-07-13T20:09:29.427054950Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-srn7t
2026-07-13T20:09:29.427108465Z insights-runtime-extractor-8skhc
2026-07-13T20:09:29.427128790Z insights-runtime-extractor-hjsdl
2026-07-13T20:09:29.427147322Z insights-runtime-extractor-twznp
2026-07-13T20:09:29.477434336Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-13T20:09:29.513325351Z INFO: Worker host service log collection to complete.
2026-07-13T20:09:29.548864905Z ERROR: No running kube-apiserver pods found
2026-07-13T20:09:29.635215283Z Wrote inspect data to must-gather.
2026-07-13T20:09:29.715563269Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-13T20:09:29.753912068Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-13T20:09:29.753912068Z See 'oc rsync -h' for help and examples
2026-07-13T20:09:29.874063822Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-13T20:09:29.987330224Z Wrote inspect data to must-gather.
2026-07-13T20:09:30.011472967Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T20:09:30.077223620Z INFO: 'previous-logs' folder not found on ip-10-0-128-174.ec2.internal, skipping...
2026-07-13T20:09:30.129064341Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-13T20:09:30.212035737Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-13T20:09:30.304854560Z tar: Removing leading `/' from member names
2026-07-13T20:09:30.311007766Z INFO: HAProxy config collection complete.
2026-07-13T20:09:30.315927681Z Wrote inspect data to must-gather.
2026-07-13T20:09:30.316756830Z Gathering data for ns/openshift-monitoring...
2026-07-13T20:09:30.330397851Z Wrote inspect data to must-gather.
2026-07-13T20:09:30.375202063Z No resources found
2026-07-13T20:09:30.463847617Z Wrote inspect data to must-gather.
2026-07-13T20:09:30.481562378Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T20:09:30.481795165Z error: the server doesn't have a resource type "machineconfigs"
2026-07-13T20:09:30.615512968Z Wrote inspect data to must-gather.
2026-07-13T20:09:30.695414509Z INFO: 'previous-logs' folder not found on ip-10-0-129-161.ec2.internal, skipping...
2026-07-13T20:09:30.717755849Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-13T20:09:30.762929412Z Wrote inspect data to must-gather.
2026-07-13T20:09:30.879870326Z INFO: Getting rules from prometheus-k8s-0
2026-07-13T20:09:30.960303637Z INFO: 'previous-logs' folder not found on ip-10-0-141-94.ec2.internal, skipping...
2026-07-13T20:09:30.960343525Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-13T20:09:30.960376568Z INFO: Machine Config Daemon termination log collection complete.
2026-07-13T20:09:30.969166106Z Wrote inspect data to must-gather.
2026-07-13T20:09:31.036568308Z Wrote inspect data to must-gather.
2026-07-13T20:09:31.113821222Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-13T20:09:31.276939928Z INFO: Getting status/config from prometheus-k8s-0
2026-07-13T20:09:31.281540585Z Wrote inspect data to must-gather.
2026-07-13T20:09:31.328951792Z Wrote inspect data to must-gather.
2026-07-13T20:09:31.360630246Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-13T20:09:31.537855740Z Wrote inspect data to must-gather.
2026-07-13T20:09:31.540579414Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:31.578702344Z INFO: INTERCONNECT MODE
2026-07-13T20:09:31.578796426Z INFO: Gathering ovn-kubernetes DBs
2026-07-13T20:09:31.677942288Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-13T20:09:31.730663613Z INFO: Gathering OVN_Northbound from ovnkube-node-8t4tr...
2026-07-13T20:09:31.732365948Z INFO: Gathering OVN_Northbound from ovnkube-node-hdsd6...
2026-07-13T20:09:31.732600373Z INFO: Gathering OVN_Northbound from ovnkube-node-nr22s...
2026-07-13T20:09:31.732937428Z INFO: Gathering OVN_Southbound from ovnkube-node-8t4tr...
2026-07-13T20:09:31.733208708Z INFO: Gathering OVN_Southbound from ovnkube-node-hdsd6...
2026-07-13T20:09:31.733511530Z INFO: Gathering OVN_Southbound from ovnkube-node-nr22s...
2026-07-13T20:09:31.772207567Z Wrote inspect data to must-gather.
2026-07-13T20:09:31.779089472Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:31.785756301Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-13T20:09:32.021092880Z tar: Removing leading `/' from member names
2026-07-13T20:09:32.034233058Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-13T20:09:32.037793897Z tar: Removing leading `/' from member names
2026-07-13T20:09:32.072868900Z tar: Removing leading `/' from member names
2026-07-13T20:09:32.084715157Z tar: Removing leading `/' from member names
2026-07-13T20:09:32.096651387Z tar: Removing leading `/' from member names
2026-07-13T20:09:32.097455760Z tar: Removing leading `/' from member names
2026-07-13T20:09:32.132486157Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-13T20:09:32.142801362Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:32.149721598Z Wrote inspect data to must-gather.
2026-07-13T20:09:32.279107367Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-13T20:09:32.363946687Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:32.366379720Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-13T20:09:32.401001734Z Wrote inspect data to must-gather.
2026-07-13T20:09:32.563578667Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-13T20:09:32.577451922Z INFO: Waiting for network log collection to complete ...
2026-07-13T20:09:32.577506257Z INFO: Waiting for ovnk database copies to complete ...
2026-07-13T20:09:32.578114951Z INFO: Copying ovnk databases complete.
2026-07-13T20:09:32.580182965Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-13T20:09:32.583562039Z ovnk_database_store/
2026-07-13T20:09:32.583609528Z ovnk_database_store/ovnkube-node-nr22s_sbdb
2026-07-13T20:09:32.599662171Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:32.603749204Z ovnk_database_store/ovnkube-node-8t4tr_sbdb
2026-07-13T20:09:32.618704160Z ovnk_database_store/ovnkube-node-8t4tr_nbdb
2026-07-13T20:09:32.625392517Z ovnk_database_store/ovnkube-node-nr22s_nbdb
2026-07-13T20:09:32.632621088Z ovnk_database_store/ovnkube-node-hdsd6_sbdb
2026-07-13T20:09:32.632876427Z Wrote inspect data to must-gather.
2026-07-13T20:09:32.648891185Z ovnk_database_store/ovnkube-node-hdsd6_nbdb
2026-07-13T20:09:32.658270234Z INFO: Network log collection complete.
2026-07-13T20:09:32.787403336Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-13T20:09:32.804300373Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:32.806953651Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-13T20:09:32.901032831Z Wrote inspect data to must-gather.
2026-07-13T20:09:32.968540477Z INFO: Getting status from alertmanager-main-0
2026-07-13T20:09:32.973672699Z Gathering data for ns/openshift-network-console...
2026-07-13T20:09:33.044811645Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:33.174789292Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-13T20:09:33.177615533Z INFO: Fallback to identify the container image from release info
2026-07-13T20:09:33.246040324Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:33.249098690Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-13T20:09:33.413107822Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:33.436927111Z Gathering data for ns/openshift-console-operator...
2026-07-13T20:09:33.591418200Z Wrote inspect data to /must-gather/istio.
2026-07-13T20:09:33.822092843Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-13T20:09:33.831467744Z Gathering data for ns/openshift-console...
2026-07-13T20:09:33.952151529Z daemonset.apps/perf-node-gather-daemonset created
2026-07-13T20:09:34.074629513Z Waiting for performance profile collector pods to become ready: 1
2026-07-13T20:09:34.263619579Z Done executing Istio gather script
2026-07-13T20:09:34.383482885Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-13T20:09:34.920986062Z Gathering data for ns/openshift-dns-operator...
2026-07-13T20:09:35.195093327Z Waiting for performance profile collector pods to become ready: 2
2026-07-13T20:09:35.206679522Z Gathering data for ns/openshift-dns...
2026-07-13T20:09:35.487042053Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T20:09:35.968248933Z Gathering data for ns/openshift-image-registry...
2026-07-13T20:09:36.301556917Z Waiting for performance profile collector pods to become ready: 3
2026-07-13T20:09:36.446374305Z Gathering data for ns/openshift-ingress-operator...
2026-07-13T20:09:36.770920038Z Gathering data for ns/openshift-ingress...
2026-07-13T20:09:37.098321993Z Gathering data for ns/openshift-ingress-canary...
2026-07-13T20:09:37.422661804Z Waiting for performance profile collector pods to become ready: 4
2026-07-13T20:09:37.483254715Z Gathering data for ns/openshift-insights...
2026-07-13T20:09:38.525696682Z Waiting for performance profile collector pods to become ready: 5
2026-07-13T20:09:39.297153518Z Gathering data for ns/openshift-config...
2026-07-13T20:09:39.624363868Z Gathering data for ns/openshift-config-managed...
2026-07-13T20:09:39.650670088Z Waiting for performance profile collector pods to become ready: 6
2026-07-13T20:09:40.087305338Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-13T20:09:40.351692478Z Gathering data for ns/openshift-kube-apiserver...
2026-07-13T20:09:40.491561156Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T20:09:40.619986296Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-13T20:09:40.765844370Z Waiting for performance profile collector pods to become ready: 7
2026-07-13T20:09:40.901216265Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-13T20:09:41.171646330Z Gathering data for ns/openshift-kube-scheduler...
2026-07-13T20:09:41.437719882Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-13T20:09:41.734752773Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-13T20:09:41.881253117Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-13T20:09:42.083477714Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-13T20:09:42.129214241Z Collecting performance related data for node ip-10-0-141-94.ec2.internal
2026-07-13T20:09:42.137970818Z Collecting performance related data for node ip-10-0-128-174.ec2.internal
2026-07-13T20:09:42.146673428Z Collecting performance related data for node ip-10-0-129-161.ec2.internal
2026-07-13T20:09:42.448589107Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-13T20:09:43.066256843Z Gathering data for ns/openshift-multus...
2026-07-13T20:09:44.609904450Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-13T20:09:45.497427370Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T20:09:45.900912053Z Gathering data for ns/openshift-host-network...
2026-07-13T20:09:46.232779289Z Gathering data for ns/openshift-network-diagnostics...
2026-07-13T20:09:46.801926835Z Gathering data for ns/openshift-network-node-identity...
2026-07-13T20:09:47.272216567Z Gathering data for ns/openshift-network-operator...
2026-07-13T20:09:47.693562455Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-13T20:09:48.000426434Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-13T20:09:48.501744042Z Gathering data for ns/openshift-apiserver-operator...
2026-07-13T20:09:48.807727487Z Gathering data for ns/openshift-apiserver...
2026-07-13T20:09:49.241180946Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-13T20:09:49.548889440Z Gathering data for ns/openshift-controller-manager...
2026-07-13T20:09:49.863548578Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-13T20:09:49.891916462Z Collecting kubelet logs for node ip-10-0-141-94.ec2.internal
2026-07-13T20:09:49.893661601Z Collecting kubelet logs for node ip-10-0-128-174.ec2.internal
2026-07-13T20:09:49.895523553Z Collecting kubelet logs for node ip-10-0-129-161.ec2.internal
2026-07-13T20:09:50.223155357Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-pvldn namespace
2026-07-13T20:09:50.228493293Z INFO: Node performance data collection complete.
2026-07-13T20:09:50.502048953Z [disk usage checker] Volume usage percentage: current = 10 ; allowed = 70
2026-07-13T20:09:50.631521420Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-13T20:09:50.933503113Z Gathering data for ns/openshift-service-ca-operator...
2026-07-13T20:09:51.288577818Z Gathering data for ns/openshift-service-ca...
2026-07-13T20:09:51.702914041Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-13T20:09:52.592257278Z Wrote inspect data to must-gather.
2026-07-13T20:09:52.592304691Z error: inspection completed with the errors occurred while gathering data:
2026-07-13T20:09:52.592304691Z     [skipping gathering secrets/support due to error: secrets "support" not found, skipping gathering daemonsets.apps/ovn-ipsec-containerized due to error: daemonsets.apps "ovn-ipsec-containerized" not found]
2026-07-13T20:09:52.714972079Z error: the server doesn't have a resource type "clusters"
2026-07-13T20:09:52.840034406Z Caches written to disk