2026-07-30T09:45:46.286386396Z [disk usage checker] Started
2026-07-30T09:45:46.289660153Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T09:45:46.472078780Z Gathering data for ns/openshift-cluster-version...
2026-07-30T09:45:46.589257882Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T09:45:46.671199713Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T09:45:46.746525097Z Gathering data for ns/default...
2026-07-30T09:45:47.036468114Z Gathering data for ns/openshift...
2026-07-30T09:45:47.437473819Z Gathering data for ns/kube-system...
2026-07-30T09:45:48.104634063Z Gathering data for ns/openshift-etcd...
2026-07-30T09:45:48.383678925Z Wrote inspect data to must-gather.
2026-07-30T09:45:48.383717416Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T09:45:48.383717416Z     namespaces "assisted-installer" not found
2026-07-30T09:45:48.853356741Z Waiting on subprocesses to finish execution.
2026-07-30T09:45:48.863624954Z Executing Istio gather script
2026-07-30T09:45:48.864056610Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-30T09:45:48.873486998Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-30T09:45:48.882130739Z INFO: Gathering HAProxy config files
2026-07-30T09:45:48.882229416Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-30T09:45:48.886402579Z INFO: Collecting host service logs for crio
2026-07-30T09:45:48.886786095Z INFO: Collecting host service logs for kubelet
2026-07-30T09:45:48.887128346Z INFO: Collecting host service logs for rpm-ostreed
2026-07-30T09:45:48.887464440Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-30T09:45:48.887771110Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-30T09:45:48.888085778Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-30T09:45:48.888442982Z INFO: Collecting host service logs for NetworkManager
2026-07-30T09:45:48.888763787Z INFO: Collecting host service logs for openvswitch
2026-07-30T09:45:48.889075098Z INFO: Collecting host service logs for ovs-configuration
2026-07-30T09:45:48.889390368Z INFO: Collecting host service logs for ovsdb-server
2026-07-30T09:45:48.889702482Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-30T09:45:48.890152965Z INFO: Waiting for worker host service log collection to complete ...
2026-07-30T09:45:48.928768005Z INFO: Waiting for node performance related collection to complete ...
2026-07-30T09:45:49.359056289Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-vw5hf
2026-07-30T09:45:49.359126681Z insights-runtime-extractor-lm9w6
2026-07-30T09:45:49.359148055Z insights-runtime-extractor-pjfgk
2026-07-30T09:45:49.359165209Z insights-runtime-extractor-vrv27
2026-07-30T09:45:49.366047396Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-30T09:45:49.639049373Z No resources found in openshift-etcd namespace.
2026-07-30T09:45:50.085859917Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-30T09:45:50.085940552Z INFO: on-disk MachineConfig config collection complete.
2026-07-30T09:45:50.087763989Z Wrote inspect data to must-gather.
2026-07-30T09:45:50.298488302Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-30T09:45:50.298488302Z See 'oc rsync -h' for help and examples
2026-07-30T09:45:50.319711952Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-30T09:45:50.450669497Z ERROR: No running kube-apiserver pods found
2026-07-30T09:45:50.477050266Z No resources found
2026-07-30T09:45:50.508461166Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-30T09:45:50.733370653Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-30T09:45:50.880776556Z INFO: "metallb-operator" not detected. Skipping.
2026-07-30T09:45:50.908722064Z Wrote inspect data to must-gather.
2026-07-30T09:45:50.931833510Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T09:45:51.001759876Z Gathering data for ns/openshift-monitoring...
2026-07-30T09:45:51.074569923Z INFO: Worker host service log collection to complete.
2026-07-30T09:45:51.093587001Z error: the server doesn't have a resource type "performanceprofile"
2026-07-30T09:45:51.110266704Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-30T09:45:51.266301608Z Wrote inspect data to must-gather.
2026-07-30T09:45:51.266496573Z No resources found
2026-07-30T09:45:51.307132692Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T09:45:51.383314050Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-30T09:45:51.400607189Z Wrote inspect data to must-gather.
2026-07-30T09:45:51.436017531Z Wrote inspect data to must-gather.
2026-07-30T09:45:51.486253692Z INFO: 'previous-logs' folder not found on ip-10-0-132-220.ec2.internal, skipping...
2026-07-30T09:45:51.501248320Z Wrote inspect data to must-gather.
2026-07-30T09:45:51.502024275Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-30T09:45:51.633798416Z Wrote inspect data to must-gather.
2026-07-30T09:45:51.693498474Z tar: Removing leading `/' from member names
2026-07-30T09:45:51.701393673Z INFO: HAProxy config collection complete.
2026-07-30T09:45:51.759442044Z Wrote inspect data to must-gather.
2026-07-30T09:45:51.813995183Z INFO: 'previous-logs' folder not found on ip-10-0-134-78.ec2.internal, skipping...
2026-07-30T09:45:51.843140387Z error: the server doesn't have a resource type "machineconfigs"
2026-07-30T09:45:51.962181462Z INFO: Getting rules from prometheus-k8s-0
2026-07-30T09:45:51.981206734Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-30T09:45:52.053929725Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-30T09:45:52.066940551Z Wrote inspect data to must-gather.
2026-07-30T09:45:52.123542912Z INFO: 'previous-logs' folder not found on ip-10-0-137-245.ec2.internal, skipping...
2026-07-30T09:45:52.123576908Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-30T09:45:52.123576908Z INFO: Machine Config Daemon termination log collection complete.
2026-07-30T09:45:52.274658621Z Wrote inspect data to must-gather.
2026-07-30T09:45:52.307756510Z Wrote inspect data to must-gather.
2026-07-30T09:45:52.388723237Z INFO: Getting status/config from prometheus-k8s-0
2026-07-30T09:45:52.449401348Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-30T09:45:52.617359094Z INFO: INTERCONNECT MODE
2026-07-30T09:45:52.617397924Z INFO: Gathering ovn-kubernetes DBs
2026-07-30T09:45:52.624279214Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-30T09:45:52.673672664Z Wrote inspect data to must-gather.
2026-07-30T09:45:52.799721989Z INFO: Gathering OVN_Northbound from ovnkube-node-5p4ps...
2026-07-30T09:45:52.799956542Z INFO: Gathering OVN_Northbound from ovnkube-node-lqdzl...
2026-07-30T09:45:52.800201053Z INFO: Gathering OVN_Northbound from ovnkube-node-q2mtk...
2026-07-30T09:45:52.800540347Z INFO: Gathering OVN_Southbound from ovnkube-node-5p4ps...
2026-07-30T09:45:52.800826384Z INFO: Gathering OVN_Southbound from ovnkube-node-lqdzl...
2026-07-30T09:45:52.801096818Z INFO: Gathering OVN_Southbound from ovnkube-node-q2mtk...
2026-07-30T09:45:52.820235216Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-30T09:45:52.919455317Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-30T09:45:53.056270174Z Wrote inspect data to must-gather.
2026-07-30T09:45:53.116591828Z tar: Removing leading `/' from member names
2026-07-30T09:45:53.116661661Z tar: Removing leading `/' from member names
2026-07-30T09:45:53.126826793Z tar: Removing leading `/' from member names
2026-07-30T09:45:53.133605033Z tar: Removing leading `/' from member names
2026-07-30T09:45:53.161661638Z tar: Removing leading `/' from member names
2026-07-30T09:45:53.207953210Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:53.222376524Z tar: Removing leading `/' from member names
2026-07-30T09:45:53.278936741Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-30T09:45:53.317355306Z W0730 09:45:53.317319    1365 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-30T09:45:53.323614871Z Wrote inspect data to must-gather.
2026-07-30T09:45:53.441932313Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-30T09:45:53.460172871Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:53.464895579Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-30T09:45:53.544821258Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-30T09:45:53.667216402Z Gathering data for ns/openshift-network-console...
2026-07-30T09:45:53.705905390Z Wrote inspect data to must-gather.
2026-07-30T09:45:53.725022110Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:53.725831075Z INFO: Waiting for network log collection to complete ...
2026-07-30T09:45:53.725945907Z INFO: Waiting for ovnk database copies to complete ...
2026-07-30T09:45:53.726509389Z INFO: Copying ovnk databases complete.
2026-07-30T09:45:53.728916846Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-07-30T09:45:53.732062395Z ovnk_database_store/
2026-07-30T09:45:53.732130829Z ovnk_database_store/ovnkube-node-lqdzl_nbdb
2026-07-30T09:45:53.740320210Z ovnk_database_store/ovnkube-node-lqdzl_sbdb
2026-07-30T09:45:53.758506459Z ovnk_database_store/ovnkube-node-q2mtk_nbdb
2026-07-30T09:45:53.766541188Z ovnk_database_store/ovnkube-node-5p4ps_sbdb
2026-07-30T09:45:53.789624692Z ovnk_database_store/ovnkube-node-q2mtk_sbdb
2026-07-30T09:45:53.804093860Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-30T09:45:53.809768788Z ovnk_database_store/ovnkube-node-5p4ps_nbdb
2026-07-30T09:45:53.821311740Z INFO: Network log collection complete.
2026-07-30T09:45:53.966235643Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:53.968953301Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-30T09:45:54.029633990Z Wrote inspect data to must-gather.
2026-07-30T09:45:54.155048425Z Gathering data for ns/openshift-console-operator...
2026-07-30T09:45:54.176255293Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:54.205519455Z INFO: Getting status from alertmanager-main-0
2026-07-30T09:45:54.359685477Z Wrote inspect data to must-gather.
2026-07-30T09:45:54.388741897Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:54.391255872Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-30T09:45:54.615416228Z Gathering data for ns/openshift-console...
2026-07-30T09:45:54.632351545Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:54.657704003Z Wrote inspect data to must-gather.
2026-07-30T09:45:54.823881837Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:54.826318861Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-30T09:45:54.909583406Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-30T09:45:54.912095666Z INFO: Fallback to identify the container image from release info
2026-07-30T09:45:54.992129664Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:55.154431302Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-30T09:45:55.181248342Z Wrote inspect data to /must-gather/istio.
2026-07-30T09:45:55.658035492Z Gathering data for ns/openshift-dns-operator...
2026-07-30T09:45:55.659623089Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-30T09:45:55.797544990Z daemonset.apps/perf-node-gather-daemonset created
2026-07-30T09:45:55.885990336Z Done executing Istio gather script
2026-07-30T09:45:55.940634353Z Waiting for performance profile collector pods to become ready: 1
2026-07-30T09:45:55.964859379Z Gathering data for ns/openshift-dns...
2026-07-30T09:45:56.312331477Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T09:45:56.562782567Z Gathering data for ns/openshift-image-registry...
2026-07-30T09:45:57.061407051Z Waiting for performance profile collector pods to become ready: 2
2026-07-30T09:45:57.067271884Z Gathering data for ns/openshift-ingress-operator...
2026-07-30T09:45:57.354278335Z Gathering data for ns/openshift-ingress...
2026-07-30T09:45:57.651238255Z Gathering data for ns/openshift-ingress-canary...
2026-07-30T09:45:57.988480016Z Gathering data for ns/openshift-insights...
2026-07-30T09:45:58.163471446Z Waiting for performance profile collector pods to become ready: 3
2026-07-30T09:45:59.267545875Z Waiting for performance profile collector pods to become ready: 4
2026-07-30T09:45:59.776361726Z Gathering data for ns/openshift-jobset-operator...
2026-07-30T09:46:00.379516707Z Waiting for performance profile collector pods to become ready: 5
2026-07-30T09:46:00.557602840Z Gathering data for ns/openshift-config...
2026-07-30T09:46:00.871601406Z Gathering data for ns/openshift-config-managed...
2026-07-30T09:46:01.251207838Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-30T09:46:01.317003586Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T09:46:01.502914289Z Waiting for performance profile collector pods to become ready: 6
2026-07-30T09:46:01.510861954Z Gathering data for ns/openshift-kube-apiserver...
2026-07-30T09:46:01.745810385Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-30T09:46:02.018206225Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-30T09:46:02.326441932Z Gathering data for ns/openshift-kube-scheduler...
2026-07-30T09:46:02.557799260Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-30T09:46:02.632302760Z Waiting for performance profile collector pods to become ready: 7
2026-07-30T09:46:02.877379694Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-30T09:46:03.168084463Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-30T09:46:03.465483498Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-30T09:46:03.753028613Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-30T09:46:03.946425496Z Gathering data for ns/openshift-multus...
2026-07-30T09:46:04.013275006Z Collecting performance related data for node ip-10-0-134-78.ec2.internal
2026-07-30T09:46:04.020581071Z Collecting performance related data for node ip-10-0-132-220.ec2.internal
2026-07-30T09:46:04.028282253Z Collecting performance related data for node ip-10-0-137-245.ec2.internal
2026-07-30T09:46:05.521206909Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-30T09:46:06.322938517Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-07-30T09:46:06.766031287Z Gathering data for ns/openshift-host-network...
2026-07-30T09:46:07.106335480Z Gathering data for ns/openshift-network-diagnostics...
2026-07-30T09:46:07.612828209Z Gathering data for ns/openshift-network-node-identity...
2026-07-30T09:46:08.034629664Z Gathering data for ns/openshift-network-operator...
2026-07-30T09:46:08.446512200Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-30T09:46:08.710271072Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-30T09:46:09.199474179Z Gathering data for ns/openshift-apiserver-operator...
2026-07-30T09:46:09.476475990Z Gathering data for ns/openshift-apiserver...
2026-07-30T09:46:09.831169434Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-30T09:46:10.062764707Z Gathering data for ns/openshift-controller-manager...
2026-07-30T09:46:10.340971900Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-30T09:46:10.964191183Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-30T09:46:11.263407021Z Gathering data for ns/openshift-service-ca-operator...
2026-07-30T09:46:11.329817127Z [disk usage checker] Volume usage percentage: current = 19 ; allowed = 70
2026-07-30T09:46:11.542189284Z Gathering data for ns/openshift-service-ca...
2026-07-30T09:46:11.876041366Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-30T09:46:12.292394448Z Collecting kubelet logs for node ip-10-0-134-78.ec2.internal
2026-07-30T09:46:12.294275763Z Collecting kubelet logs for node ip-10-0-132-220.ec2.internal
2026-07-30T09:46:12.296161502Z Collecting kubelet logs for node ip-10-0-137-245.ec2.internal
2026-07-30T09:46:12.622645226Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-vzxn6 namespace
2026-07-30T09:46:12.626482406Z INFO: Node performance data collection complete.
2026-07-30T09:46:12.819951714Z Wrote inspect data to must-gather.
2026-07-30T09:46:12.819996157Z error: inspection completed with the errors occurred while gathering data:
2026-07-30T09:46:12.819996157Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-30T09:46:12.949123777Z error: the server doesn't have a resource type "clusters"
2026-07-30T09:46:13.081263885Z Caches written to disk