2026-07-31T16:55:11.551511454Z [disk usage checker] Started
2026-07-31T16:55:11.555388473Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-31T16:55:11.737644542Z Gathering data for ns/openshift-cluster-version...
2026-07-31T16:55:11.873425374Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T16:55:11.955244558Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T16:55:12.031175590Z Gathering data for ns/default...
2026-07-31T16:55:12.289101497Z Gathering data for ns/openshift...
2026-07-31T16:55:12.668059735Z Gathering data for ns/kube-system...
2026-07-31T16:55:13.291093782Z Gathering data for ns/openshift-etcd...
2026-07-31T16:55:13.553288136Z Wrote inspect data to must-gather.
2026-07-31T16:55:13.553318952Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T16:55:13.553318952Z     namespaces "assisted-installer" not found
2026-07-31T16:55:14.000917630Z Waiting on subprocesses to finish execution.
2026-07-31T16:55:14.010160018Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-31T16:55:14.021662698Z Executing Istio gather script
2026-07-31T16:55:14.036055822Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-31T16:55:14.046741074Z INFO: Gathering HAProxy config files
2026-07-31T16:55:14.047325340Z INFO: Collecting host service logs for crio
2026-07-31T16:55:14.047641538Z INFO: Collecting host service logs for kubelet
2026-07-31T16:55:14.047954516Z INFO: Collecting host service logs for rpm-ostreed
2026-07-31T16:55:14.048229813Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-31T16:55:14.048510675Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-31T16:55:14.048801336Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-31T16:55:14.049075654Z INFO: Collecting host service logs for NetworkManager
2026-07-31T16:55:14.049340798Z INFO: Collecting host service logs for openvswitch
2026-07-31T16:55:14.049611193Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-31T16:55:14.049624046Z INFO: Collecting host service logs for ovs-configuration
2026-07-31T16:55:14.049919442Z INFO: Collecting host service logs for ovsdb-server
2026-07-31T16:55:14.050192834Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-31T16:55:14.050526672Z INFO: Waiting for worker host service log collection to complete ...
2026-07-31T16:55:14.074861078Z INFO: Waiting for node performance related collection to complete ...
2026-07-31T16:55:14.786862512Z INFO: "metallb-operator" not detected. Skipping.
2026-07-31T16:55:14.840062314Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-31T16:55:14.904456772Z No resources found in openshift-etcd namespace.
2026-07-31T16:55:14.936916811Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-31T16:55:14.936954363Z INFO: on-disk MachineConfig config collection complete.
2026-07-31T16:55:15.014264607Z error: the server doesn't have a resource type "performanceprofile"
2026-07-31T16:55:15.461379176Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-31T16:55:15.489065755Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-31T16:55:15.573280768Z No resources found
2026-07-31T16:55:15.674334819Z INFO: Collecting Insights Archives from insights-operator-77696685b-qqr52
2026-07-31T16:55:15.674334819Z insights-runtime-extractor-j9zh2
2026-07-31T16:55:15.674334819Z insights-runtime-extractor-nlvhp
2026-07-31T16:55:15.674334819Z insights-runtime-extractor-w6cbk
2026-07-31T16:55:15.707744422Z Wrote inspect data to must-gather.
2026-07-31T16:55:15.767632446Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-31T16:55:15.927315365Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T16:55:15.950814887Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.037153537Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.062424437Z No resources found
2026-07-31T16:55:16.107607191Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-31T16:55:16.117780540Z INFO: Worker host service log collection to complete.
2026-07-31T16:55:16.174986933Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.266730172Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-31T16:55:16.266730172Z See 'oc rsync -h' for help and examples
2026-07-31T16:55:16.280583950Z ERROR: No running kube-apiserver pods found
2026-07-31T16:55:16.450679093Z Gathering data for ns/openshift-monitoring...
2026-07-31T16:55:16.451625683Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.470450085Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.502854142Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.529150008Z INFO: 'previous-logs' folder not found on ip-10-0-130-3.ec2.internal, skipping...
2026-07-31T16:55:16.547442388Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-31T16:55:16.563325352Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-31T16:55:16.582085011Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-31T16:55:16.612696594Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.733934685Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-31T16:55:16.798651008Z error: the server doesn't have a resource type "machineconfigs"
2026-07-31T16:55:16.809499808Z Wrote inspect data to must-gather.
2026-07-31T16:55:16.833969003Z tar: Removing leading `/' from member names
2026-07-31T16:55:16.835862240Z INFO: 'previous-logs' folder not found on ip-10-0-134-150.ec2.internal, skipping...
2026-07-31T16:55:16.837982584Z INFO: HAProxy config collection complete.
2026-07-31T16:55:16.974960084Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-31T16:55:17.049327592Z INFO: Getting rules from prometheus-k8s-0
2026-07-31T16:55:17.086413045Z Wrote inspect data to must-gather.
2026-07-31T16:55:17.124235843Z INFO: 'previous-logs' folder not found on ip-10-0-135-235.ec2.internal, skipping...
2026-07-31T16:55:17.124271151Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-31T16:55:17.124278899Z INFO: Machine Config Daemon termination log collection complete.
2026-07-31T16:55:17.214708425Z Wrote inspect data to must-gather.
2026-07-31T16:55:17.362924538Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-31T16:55:17.370485718Z INFO: INTERCONNECT MODE
2026-07-31T16:55:17.370512971Z INFO: Gathering ovn-kubernetes DBs
2026-07-31T16:55:17.463777518Z INFO: Getting status/config from prometheus-k8s-0
2026-07-31T16:55:17.538816655Z INFO: Gathering OVN_Northbound from ovnkube-node-k7kgh...
2026-07-31T16:55:17.539154853Z INFO: Gathering OVN_Northbound from ovnkube-node-nb45k...
2026-07-31T16:55:17.539428118Z INFO: Gathering OVN_Northbound from ovnkube-node-xpv4v...
2026-07-31T16:55:17.539962427Z INFO: Gathering OVN_Southbound from ovnkube-node-k7kgh...
2026-07-31T16:55:17.540318722Z INFO: Gathering OVN_Southbound from ovnkube-node-nb45k...
2026-07-31T16:55:17.540599430Z INFO: Gathering OVN_Southbound from ovnkube-node-xpv4v...
2026-07-31T16:55:17.580748323Z Wrote inspect data to must-gather.
2026-07-31T16:55:17.646040964Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-31T16:55:17.822936039Z tar: Removing leading `/' from member names
2026-07-31T16:55:17.854947906Z tar: Removing leading `/' from member names
2026-07-31T16:55:17.884290923Z Wrote inspect data to must-gather.
2026-07-31T16:55:17.887052136Z tar: Removing leading `/' from member names
2026-07-31T16:55:17.898423871Z tar: Removing leading `/' from member names
2026-07-31T16:55:17.908325728Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:17.912897675Z tar: Removing leading `/' from member names
2026-07-31T16:55:17.915262698Z tar: Removing leading `/' from member names
2026-07-31T16:55:17.943030014Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-31T16:55:18.119522709Z Wrote inspect data to must-gather.
2026-07-31T16:55:18.159032896Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:18.162568311Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-31T16:55:18.207232668Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-31T16:55:18.352822256Z INFO: Waiting for network log collection to complete ...
2026-07-31T16:55:18.352863751Z INFO: Waiting for ovnk database copies to complete ...
2026-07-31T16:55:18.353253069Z INFO: Copying ovnk databases complete.
2026-07-31T16:55:18.354912400Z 2.5M	must-gather/network_logs/ovnk_database_store
2026-07-31T16:55:18.356994044Z ovnk_database_store/
2026-07-31T16:55:18.356994044Z ovnk_database_store/ovnkube-node-k7kgh_nbdb
2026-07-31T16:55:18.360162137Z Wrote inspect data to must-gather.
2026-07-31T16:55:18.363091719Z ovnk_database_store/ovnkube-node-nb45k_sbdb
2026-07-31T16:55:18.369452099Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-31T16:55:18.379921475Z ovnk_database_store/ovnkube-node-xpv4v_sbdb
2026-07-31T16:55:18.395368076Z ovnk_database_store/ovnkube-node-nb45k_nbdb
2026-07-31T16:55:18.395396537Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:18.402973584Z ovnk_database_store/ovnkube-node-xpv4v_nbdb
2026-07-31T16:55:18.410257881Z ovnk_database_store/ovnkube-node-k7kgh_sbdb
2026-07-31T16:55:18.430583504Z INFO: Network log collection complete.
2026-07-31T16:55:18.481042827Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-31T16:55:18.565861556Z Wrote inspect data to must-gather.
2026-07-31T16:55:18.601930412Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:18.606722346Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-31T16:55:18.716967080Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-31T16:55:18.791704126Z Wrote inspect data to must-gather.
2026-07-31T16:55:18.822280780Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:18.996895424Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-31T16:55:19.040922239Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:19.044194033Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-31T16:55:19.089948559Z Wrote inspect data to must-gather.
2026-07-31T16:55:19.110214735Z INFO: Getting status from alertmanager-main-0
2026-07-31T16:55:19.191607646Z Gathering data for ns/openshift-network-console...
2026-07-31T16:55:19.284399570Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:19.354836861Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-31T16:55:19.356647150Z INFO: Fallback to identify the container image from release info
2026-07-31T16:55:19.471681209Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:19.476175138Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-31T16:55:19.636420459Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:19.658211226Z Gathering data for ns/openshift-console-operator...
2026-07-31T16:55:19.817828996Z Wrote inspect data to /must-gather/istio.
2026-07-31T16:55:20.018170435Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f2eb1033f55d7129b37ecfc3e4e9a306e42e5ab3dd7d2fe0ff08c8b45ea2b675
2026-07-31T16:55:20.070611621Z Gathering data for ns/openshift-console...
2026-07-31T16:55:20.141444979Z daemonset.apps/perf-node-gather-daemonset created
2026-07-31T16:55:20.283148459Z Waiting for performance profile collector pods to become ready: 1
2026-07-31T16:55:20.471813032Z Done executing Istio gather script
2026-07-31T16:55:20.592100344Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-31T16:55:21.062993678Z Gathering data for ns/openshift-dns-operator...
2026-07-31T16:55:21.330897836Z Gathering data for ns/openshift-dns...
2026-07-31T16:55:21.387304790Z Waiting for performance profile collector pods to become ready: 2
2026-07-31T16:55:21.587992509Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-31T16:55:21.920108147Z Gathering data for ns/openshift-image-registry...
2026-07-31T16:55:22.367096739Z Gathering data for ns/openshift-ingress-operator...
2026-07-31T16:55:22.495029266Z Waiting for performance profile collector pods to become ready: 3
2026-07-31T16:55:22.680247104Z Gathering data for ns/openshift-ingress...
2026-07-31T16:55:23.008441798Z Gathering data for ns/openshift-ingress-canary...
2026-07-31T16:55:23.360760438Z Gathering data for ns/openshift-insights...
2026-07-31T16:55:23.594604733Z Waiting for performance profile collector pods to become ready: 4
2026-07-31T16:55:24.700386941Z Waiting for performance profile collector pods to become ready: 5
2026-07-31T16:55:25.127180102Z Gathering data for ns/openshift-config...
2026-07-31T16:55:25.378277869Z Gathering data for ns/openshift-config-managed...
2026-07-31T16:55:25.754123809Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-31T16:55:25.832251134Z Waiting for performance profile collector pods to become ready: 6
2026-07-31T16:55:26.004211316Z Gathering data for ns/openshift-kube-apiserver...
2026-07-31T16:55:26.243541202Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-31T16:55:26.499929518Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-31T16:55:26.592575901Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-31T16:55:26.766852735Z Gathering data for ns/openshift-kube-scheduler...
2026-07-31T16:55:26.946591024Z Waiting for performance profile collector pods to become ready: 7
2026-07-31T16:55:27.003319928Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-31T16:55:27.267937622Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-31T16:55:27.609239618Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-31T16:55:27.918050544Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-31T16:55:28.056454176Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-31T16:55:28.293937304Z Collecting performance related data for node ip-10-0-135-235.ec2.internal
2026-07-31T16:55:28.301164799Z Collecting performance related data for node ip-10-0-130-3.ec2.internal
2026-07-31T16:55:28.309838084Z Collecting performance related data for node ip-10-0-134-150.ec2.internal
2026-07-31T16:55:28.528980859Z Gathering data for ns/openshift-multus...
2026-07-31T16:55:30.080595390Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-31T16:55:31.288558825Z Gathering data for ns/openshift-host-network...
2026-07-31T16:55:31.558538483Z Gathering data for ns/openshift-network-diagnostics...
2026-07-31T16:55:31.597323511Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-31T16:55:32.043486378Z Gathering data for ns/openshift-network-node-identity...
2026-07-31T16:55:32.416993541Z Gathering data for ns/openshift-network-operator...
2026-07-31T16:55:32.776691477Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-31T16:55:33.008875362Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-31T16:55:33.458944614Z Gathering data for ns/openshift-apiserver-operator...
2026-07-31T16:55:33.737008318Z Gathering data for ns/openshift-apiserver...
2026-07-31T16:55:34.091825130Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-31T16:55:34.330895231Z Gathering data for ns/openshift-controller-manager...
2026-07-31T16:55:34.568906599Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-31T16:55:35.195099163Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-31T16:55:35.486520124Z Gathering data for ns/openshift-service-ca-operator...
2026-07-31T16:55:35.761213323Z Gathering data for ns/openshift-service-ca...
2026-07-31T16:55:36.116172988Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-31T16:55:36.247411772Z Collecting kubelet logs for node ip-10-0-135-235.ec2.internal
2026-07-31T16:55:36.249125914Z Collecting kubelet logs for node ip-10-0-130-3.ec2.internal
2026-07-31T16:55:36.250887540Z Collecting kubelet logs for node ip-10-0-134-150.ec2.internal
2026-07-31T16:55:36.561555946Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-4fldk namespace
2026-07-31T16:55:36.567142494Z INFO: Node performance data collection complete.
2026-07-31T16:55:36.602507088Z [disk usage checker] Volume usage percentage: current = 14 ; allowed = 70
2026-07-31T16:55:36.996737947Z Wrote inspect data to must-gather.
2026-07-31T16:55:36.996772319Z error: inspection completed with the errors occurred while gathering data:
2026-07-31T16:55:36.996772319Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-31T16:55:37.117139614Z error: the server doesn't have a resource type "clusters"
2026-07-31T16:55:37.241332828Z Caches written to disk