2026-07-27T17:02:22.096414440Z [disk usage checker] Started
2026-07-27T17:02:22.099767083Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-27T17:02:22.302847069Z Gathering data for ns/openshift-cluster-version...
2026-07-27T17:02:22.451166298Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-27T17:02:22.543671989Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T17:02:22.625658153Z Gathering data for ns/default...
2026-07-27T17:02:22.931338944Z Gathering data for ns/openshift...
2026-07-27T17:02:23.340580205Z Gathering data for ns/kube-system...
2026-07-27T17:02:24.077401714Z Gathering data for ns/openshift-etcd...
2026-07-27T17:02:24.378336396Z Wrote inspect data to must-gather.
2026-07-27T17:02:24.378363401Z error: inspection completed with the errors occurred while gathering data:
2026-07-27T17:02:24.378363401Z     namespaces "assisted-installer" not found
2026-07-27T17:02:24.628719093Z Waiting on subprocesses to finish execution.
2026-07-27T17:02:24.642913205Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-27T17:02:24.644342651Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-27T17:02:24.650694194Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-27T17:02:24.654690525Z INFO: Collecting host service logs for crio
2026-07-27T17:02:24.654962255Z INFO: Collecting host service logs for kubelet
2026-07-27T17:02:24.655288084Z INFO: Collecting host service logs for rpm-ostreed
2026-07-27T17:02:24.655626162Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-27T17:02:24.655964135Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-27T17:02:24.656292980Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-27T17:02:24.656644450Z INFO: Collecting host service logs for NetworkManager
2026-07-27T17:02:24.656983653Z INFO: Collecting host service logs for openvswitch
2026-07-27T17:02:24.657328181Z INFO: Collecting host service logs for ovs-configuration
2026-07-27T17:02:24.657690233Z INFO: Collecting host service logs for ovsdb-server
2026-07-27T17:02:24.658038221Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-27T17:02:24.658424293Z INFO: Waiting for worker host service log collection to complete ...
2026-07-27T17:02:24.659240548Z INFO: Gathering HAProxy config files
2026-07-27T17:02:24.666746728Z Executing Istio gather script
2026-07-27T17:02:24.694188183Z INFO: Waiting for node performance related collection to complete ...
2026-07-27T17:02:25.001613267Z INFO: "metallb-operator" not detected. Skipping.
2026-07-27T17:02:25.054090522Z No resources found in openshift-etcd namespace.
2026-07-27T17:02:25.501187477Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-27T17:02:25.501278455Z INFO: on-disk MachineConfig config collection complete.
2026-07-27T17:02:25.776372719Z error: the server doesn't have a resource type "performanceprofile"
2026-07-27T17:02:25.799806938Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-27T17:02:25.855680451Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-27T17:02:25.867527743Z No resources found
2026-07-27T17:02:25.887171110Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-27T17:02:25.922641416Z ERROR: No running kube-apiserver pods found
2026-07-27T17:02:25.965726520Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-27T17:02:25.993225210Z INFO: Worker host service log collection to complete.
2026-07-27T17:02:26.038892014Z INFO: Collecting Insights Archives from insights-operator-5885fdb759-hsfsj
2026-07-27T17:02:26.038892014Z insights-runtime-extractor-7cqsz
2026-07-27T17:02:26.038892014Z insights-runtime-extractor-7m8k7
2026-07-27T17:02:26.038892014Z insights-runtime-extractor-gwc6f
2026-07-27T17:02:26.257278960Z Wrote inspect data to must-gather.
2026-07-27T17:02:26.400371215Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-27T17:02:26.400371215Z See 'oc rsync -h' for help and examples
2026-07-27T17:02:26.537246033Z Gathering data for ns/openshift-monitoring...
2026-07-27T17:02:26.562911304Z Wrote inspect data to must-gather.
2026-07-27T17:02:26.627303560Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-27T17:02:26.730362991Z No resources found
2026-07-27T17:02:26.750674440Z INFO: 'previous-logs' folder not found on ip-10-0-131-224.ec2.internal, skipping...
2026-07-27T17:02:26.852081604Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-27T17:02:26.927037379Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T17:02:26.967842358Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.033685962Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.129556293Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.130276039Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-27T17:02:27.171154862Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.194239548Z tar: Removing leading `/' from member names
2026-07-27T17:02:27.207898205Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-27T17:02:27.209036706Z INFO: HAProxy config collection complete.
2026-07-27T17:02:27.244591740Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.288000587Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-27T17:02:27.307382696Z error: the server doesn't have a resource type "machineconfigs"
2026-07-27T17:02:27.324538841Z INFO: 'previous-logs' folder not found on ip-10-0-136-189.ec2.internal, skipping...
2026-07-27T17:02:27.480394776Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-27T17:02:27.530672852Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.588585404Z INFO: Getting rules from prometheus-k8s-0
2026-07-27T17:02:27.596902952Z INFO: 'previous-logs' folder not found on ip-10-0-141-8.ec2.internal, skipping...
2026-07-27T17:02:27.596937513Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-27T17:02:27.596937513Z INFO: Machine Config Daemon termination log collection complete.
2026-07-27T17:02:27.706447652Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.794152868Z Wrote inspect data to must-gather.
2026-07-27T17:02:27.856714406Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-27T17:02:28.023617183Z INFO: Getting status/config from prometheus-k8s-0
2026-07-27T17:02:28.044108218Z Wrote inspect data to must-gather.
2026-07-27T17:02:28.088428899Z Wrote inspect data to must-gather.
2026-07-27T17:02:28.309611979Z Wrote inspect data to must-gather.
2026-07-27T17:02:28.350541927Z INFO: INTERCONNECT MODE
2026-07-27T17:02:28.350587242Z INFO: Gathering ovn-kubernetes DBs
2026-07-27T17:02:28.458740743Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-27T17:02:28.495942644Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-27T17:02:28.523810056Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-27T17:02:28.525191280Z INFO: Gathering OVN_Northbound from ovnkube-node-7bhsq...
2026-07-27T17:02:28.525428761Z INFO: Gathering OVN_Northbound from ovnkube-node-scwfd...
2026-07-27T17:02:28.525683168Z INFO: Gathering OVN_Northbound from ovnkube-node-svr4j...
2026-07-27T17:02:28.525964391Z INFO: Gathering OVN_Southbound from ovnkube-node-7bhsq...
2026-07-27T17:02:28.526264694Z INFO: Gathering OVN_Southbound from ovnkube-node-scwfd...
2026-07-27T17:02:28.526778308Z INFO: Gathering OVN_Southbound from ovnkube-node-svr4j...
2026-07-27T17:02:28.586844029Z Wrote inspect data to must-gather.
2026-07-27T17:02:28.898330887Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-27T17:02:28.923232831Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:28.950650631Z tar: Removing leading `/' from member names
2026-07-27T17:02:28.970836132Z Wrote inspect data to must-gather.
2026-07-27T17:02:28.972053362Z tar: Removing leading `/' from member names
2026-07-27T17:02:28.981796063Z tar: Removing leading `/' from member names
2026-07-27T17:02:28.983638842Z tar: Removing leading `/' from member names
2026-07-27T17:02:28.986741160Z tar: Removing leading `/' from member names
2026-07-27T17:02:28.997600095Z tar: Removing leading `/' from member names
2026-07-27T17:02:29.192476593Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:29.196507937Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-27T17:02:29.197766146Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-27T17:02:29.233884560Z W0727 17:02:29.233840    1415 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-07-27T17:02:29.241963226Z Wrote inspect data to must-gather.
2026-07-27T17:02:29.348310166Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-27T17:02:29.434516925Z INFO: Waiting for network log collection to complete ...
2026-07-27T17:02:29.434558301Z INFO: Waiting for ovnk database copies to complete ...
2026-07-27T17:02:29.435067689Z INFO: Copying ovnk databases complete.
2026-07-27T17:02:29.437188916Z 2.8M	must-gather/network_logs/ovnk_database_store
2026-07-27T17:02:29.439855742Z ovnk_database_store/
2026-07-27T17:02:29.439855742Z ovnk_database_store/ovnkube-node-scwfd_nbdb
2026-07-27T17:02:29.449068718Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:29.449408042Z ovnk_database_store/ovnkube-node-7bhsq_sbdb
2026-07-27T17:02:29.464335355Z ovnk_database_store/ovnkube-node-7bhsq_nbdb
2026-07-27T17:02:29.473288282Z ovnk_database_store/ovnkube-node-svr4j_nbdb
2026-07-27T17:02:29.478395454Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-27T17:02:29.480682468Z ovnk_database_store/ovnkube-node-svr4j_sbdb
2026-07-27T17:02:29.500642527Z ovnk_database_store/ovnkube-node-scwfd_sbdb
2026-07-27T17:02:29.522992194Z INFO: Network log collection complete.
2026-07-27T17:02:29.563774847Z Gathering data for ns/openshift-network-console...
2026-07-27T17:02:29.581615963Z Wrote inspect data to must-gather.
2026-07-27T17:02:29.673807077Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:29.677796650Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-27T17:02:29.867049964Z INFO: Getting status from alertmanager-main-0
2026-07-27T17:02:29.881387545Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:29.890734506Z Wrote inspect data to must-gather.
2026-07-27T17:02:30.053430497Z Gathering data for ns/openshift-console-operator...
2026-07-27T17:02:30.105844579Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:30.109959991Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-27T17:02:30.169970521Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-27T17:02:30.172837014Z INFO: Fallback to identify the container image from release info
2026-07-27T17:02:30.334253948Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:30.453782503Z Gathering data for ns/openshift-console...
2026-07-27T17:02:30.521235375Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:30.523663200Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-27T17:02:30.690231488Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:30.715540339Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a83451907191f154bed1dfc7f0174433f0075beb6b6b82b9ec61e07e402dc811
2026-07-27T17:02:30.837451409Z daemonset.apps/perf-node-gather-daemonset created
2026-07-27T17:02:30.910851375Z Wrote inspect data to /must-gather/istio.
2026-07-27T17:02:30.973521025Z Waiting for performance profile collector pods to become ready: 1
2026-07-27T17:02:31.037080540Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-27T17:02:31.592262724Z Gathering data for ns/openshift-dns-operator...
2026-07-27T17:02:31.597281306Z Done executing Istio gather script
2026-07-27T17:02:31.849360657Z Gathering data for ns/openshift-dns...
2026-07-27T17:02:32.079517980Z Waiting for performance profile collector pods to become ready: 2
2026-07-27T17:02:32.136396755Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-27T17:02:32.447298321Z Gathering data for ns/openshift-image-registry...
2026-07-27T17:02:32.922635131Z Gathering data for ns/openshift-ingress-operator...
2026-07-27T17:02:33.196723976Z Waiting for performance profile collector pods to become ready: 3
2026-07-27T17:02:33.250594294Z Gathering data for ns/openshift-ingress...
2026-07-27T17:02:33.563705895Z Gathering data for ns/openshift-ingress-canary...
2026-07-27T17:02:33.951328188Z Gathering data for ns/openshift-insights...
2026-07-27T17:02:34.319516805Z Waiting for performance profile collector pods to become ready: 4
2026-07-27T17:02:35.418934695Z Waiting for performance profile collector pods to become ready: 5
2026-07-27T17:02:35.852750123Z Gathering data for ns/openshift-jobset-operator...
2026-07-27T17:02:36.524624734Z Waiting for performance profile collector pods to become ready: 6
2026-07-27T17:02:36.667052085Z Gathering data for ns/openshift-config...
2026-07-27T17:02:36.934417950Z Gathering data for ns/openshift-config-managed...
2026-07-27T17:02:37.141452611Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-27T17:02:37.340650954Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-27T17:02:37.612756923Z Gathering data for ns/openshift-kube-apiserver...
2026-07-27T17:02:37.649617758Z Waiting for performance profile collector pods to become ready: 7
2026-07-27T17:02:37.891206535Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-27T17:02:38.167892342Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-27T17:02:38.446629357Z Gathering data for ns/openshift-kube-scheduler...
2026-07-27T17:02:38.716513374Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-27T17:02:38.773660680Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-27T17:02:39.003645592Z Collecting performance related data for node ip-10-0-141-8.ec2.internal
2026-07-27T17:02:39.011456682Z Collecting performance related data for node ip-10-0-131-224.ec2.internal
2026-07-27T17:02:39.020116799Z Collecting performance related data for node ip-10-0-136-189.ec2.internal
2026-07-27T17:02:39.073896185Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-27T17:02:39.436864636Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-27T17:02:39.806004641Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-27T17:02:40.434464709Z Gathering data for ns/openshift-multus...
2026-07-27T17:02:41.988951377Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-27T17:02:42.146677635Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-27T17:02:43.299416822Z Gathering data for ns/openshift-host-network...
2026-07-27T17:02:43.625385186Z Gathering data for ns/openshift-network-diagnostics...
2026-07-27T17:02:44.176800233Z Gathering data for ns/openshift-network-node-identity...
2026-07-27T17:02:44.601481285Z Gathering data for ns/openshift-network-operator...
2026-07-27T17:02:45.025982803Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-27T17:02:45.302601462Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-27T17:02:45.811734850Z Gathering data for ns/openshift-apiserver-operator...
2026-07-27T17:02:46.146931510Z Gathering data for ns/openshift-apiserver...
2026-07-27T17:02:46.576183370Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-27T17:02:46.886780421Z Gathering data for ns/openshift-controller-manager...
2026-07-27T17:02:47.009398715Z Collecting kubelet logs for node ip-10-0-141-8.ec2.internal
2026-07-27T17:02:47.011198997Z Collecting kubelet logs for node ip-10-0-131-224.ec2.internal
2026-07-27T17:02:47.012826745Z Collecting kubelet logs for node ip-10-0-136-189.ec2.internal
2026-07-27T17:02:47.154765143Z [disk usage checker] Volume usage percentage: current = 15 ; allowed = 70
2026-07-27T17:02:47.203852327Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-27T17:02:47.421422199Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-ghnrl namespace
2026-07-27T17:02:47.426856591Z INFO: Node performance data collection complete.
2026-07-27T17:02:47.934953708Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-27T17:02:48.215392498Z Gathering data for ns/openshift-service-ca-operator...
2026-07-27T17:02:48.512900807Z Gathering data for ns/openshift-service-ca...
2026-07-27T17:02:48.911283754Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-27T17:02:49.774172708Z Wrote inspect data to must-gather.
2026-07-27T17:02:49.774215402Z error: inspection completed with the errors occurred while gathering data:
2026-07-27T17:02:49.774215402Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-27T17:02:49.894754936Z error: the server doesn't have a resource type "clusters"
2026-07-27T17:02:50.029166872Z Caches written to disk