2026-06-05T11:54:37.807187101Z [disk usage checker] Started
2026-06-05T11:54:37.810976582Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-05T11:54:38.002689397Z Gathering data for ns/openshift-cluster-version...
2026-06-05T11:54:38.154000226Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T11:54:38.249219504Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T11:54:38.328144279Z Gathering data for ns/default...
2026-06-05T11:54:38.617025998Z Gathering data for ns/openshift...
2026-06-05T11:54:39.019288291Z Gathering data for ns/kube-system...
2026-06-05T11:54:39.738544629Z Gathering data for ns/openshift-etcd...
2026-06-05T11:54:40.049223686Z Wrote inspect data to must-gather.
2026-06-05T11:54:40.049309713Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T11:54:40.049309713Z     namespaces "assisted-installer" not found
2026-06-05T11:54:40.335617220Z Waiting on subprocesses to finish execution.
2026-06-05T11:54:40.346926139Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-06-05T11:54:40.352148602Z Executing Istio gather script
2026-06-05T11:54:40.359924086Z INFO: Gathering machine config daemon's old logs from all nodes
2026-06-05T11:54:40.361321214Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-06-05T11:54:40.372580944Z INFO: Gathering HAProxy config files
2026-06-05T11:54:40.374904136Z INFO: Collecting host service logs for crio
2026-06-05T11:54:40.375226735Z INFO: Collecting host service logs for kubelet
2026-06-05T11:54:40.375598015Z INFO: Collecting host service logs for rpm-ostreed
2026-06-05T11:54:40.375943501Z INFO: Collecting host service logs for ostree-finalize-staged
2026-06-05T11:54:40.376269857Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-06-05T11:54:40.376650000Z INFO: Collecting host service logs for machine-config-daemon-host
2026-06-05T11:54:40.376987355Z INFO: Collecting host service logs for NetworkManager
2026-06-05T11:54:40.377307391Z INFO: Collecting host service logs for openvswitch
2026-06-05T11:54:40.377672481Z INFO: Collecting host service logs for ovs-configuration
2026-06-05T11:54:40.378147753Z INFO: Collecting host service logs for ovsdb-server
2026-06-05T11:54:40.378524428Z INFO: Collecting host service logs for ovs-vswitchd
2026-06-05T11:54:40.378927742Z INFO: Waiting for worker host service log collection to complete ...
2026-06-05T11:54:40.400839541Z INFO: Waiting for node performance related collection to complete ...
2026-06-05T11:54:40.599577051Z No resources found in openshift-etcd namespace.
2026-06-05T11:54:41.319327868Z INFO: Collecting Insights Archives from insights-operator-896549976-snr4d
2026-06-05T11:54:41.319384195Z insights-runtime-extractor-dqrdk
2026-06-05T11:54:41.319405541Z insights-runtime-extractor-g92sp
2026-06-05T11:54:41.319425705Z insights-runtime-extractor-zxsqw
2026-06-05T11:54:41.319866505Z INFO: "metallb-operator" not detected. Skipping.
2026-06-05T11:54:41.502239201Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-06-05T11:54:41.510137662Z No resources found
2026-06-05T11:54:41.557527101Z Wrote inspect data to must-gather.
2026-06-05T11:54:41.582599645Z Wrote inspect data to must-gather.
2026-06-05T11:54:41.680009925Z INFO: "sriov-network-operator" not detected. Skipping.
2026-06-05T11:54:41.775734156Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-06-05T11:54:41.775821102Z INFO: on-disk MachineConfig config collection complete.
2026-06-05T11:54:42.056007635Z INFO: Found 1 replicas - prometheus-k8s-0
2026-06-05T11:54:42.084093597Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-06-05T11:54:42.163632973Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-06-05T11:54:42.163632973Z See 'oc rsync -h' for help and examples
2026-06-05T11:54:42.185415526Z error: the server doesn't have a resource type "performanceprofile"
2026-06-05T11:54:42.443692268Z ERROR: No running kube-apiserver pods found
2026-06-05T11:54:42.499567441Z INFO: Worker host service log collection to complete.
2026-06-05T11:54:42.540166154Z No resources found
2026-06-05T11:54:42.561407586Z Wrote inspect data to must-gather.
2026-06-05T11:54:42.671387363Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T11:54:42.757448736Z Wrote inspect data to must-gather.
2026-06-05T11:54:42.762933106Z Wrote inspect data to must-gather.
2026-06-05T11:54:42.841630519Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-06-05T11:54:42.851898819Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-05T11:54:42.867131687Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-06-05T11:54:42.876255377Z Gathering data for ns/openshift-monitoring...
2026-06-05T11:54:42.881441915Z INFO: Waiting for HAProxy config collection to complete ...
2026-06-05T11:54:42.953549381Z Wrote inspect data to must-gather.
2026-06-05T11:54:42.968788539Z Wrote inspect data to must-gather.
2026-06-05T11:54:43.075928069Z INFO: 'previous-logs' folder not found on ip-10-0-130-222.ec2.internal, skipping...
2026-06-05T11:54:43.155848167Z error: the server doesn't have a resource type "machineconfigs"
2026-06-05T11:54:43.164796735Z Wrote inspect data to must-gather.
2026-06-05T11:54:43.167070760Z tar: Removing leading `/' from member names
2026-06-05T11:54:43.174183537Z INFO: HAProxy config collection complete.
2026-06-05T11:54:43.281749241Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-06-05T11:54:43.344603944Z error: the server doesn't have a resource type "machineconfigpools"
2026-06-05T11:54:43.374294542Z INFO: Getting rules from prometheus-k8s-0
2026-06-05T11:54:43.392194022Z INFO: 'previous-logs' folder not found on ip-10-0-130-33.ec2.internal, skipping...
2026-06-05T11:54:43.446204170Z Wrote inspect data to must-gather.
2026-06-05T11:54:43.592296330Z Wrote inspect data to must-gather.
2026-06-05T11:54:43.635387423Z INFO: 'previous-logs' folder not found on ip-10-0-138-129.ec2.internal, skipping...
2026-06-05T11:54:43.635429025Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-06-05T11:54:43.635462237Z INFO: Machine Config Daemon termination log collection complete.
2026-06-05T11:54:43.709871327Z Wrote inspect data to must-gather.
2026-06-05T11:54:43.764315009Z error: the server doesn't have a resource type "kubeletconfigs"
2026-06-05T11:54:43.776939252Z INFO: Getting status/config from prometheus-k8s-0
2026-06-05T11:54:43.972938716Z Wrote inspect data to must-gather.
2026-06-05T11:54:43.989051337Z INFO: INTERCONNECT MODE
2026-06-05T11:54:43.989079895Z INFO: Gathering ovn-kubernetes DBs
2026-06-05T11:54:44.148199398Z INFO: Gathering OVN_Northbound from ovnkube-node-5ctg8...
2026-06-05T11:54:44.148563177Z INFO: Gathering OVN_Northbound from ovnkube-node-rkx2v...
2026-06-05T11:54:44.148869740Z INFO: Gathering OVN_Northbound from ovnkube-node-vnn5f...
2026-06-05T11:54:44.149140867Z INFO: Gathering OVN_Southbound from ovnkube-node-5ctg8...
2026-06-05T11:54:44.149473342Z INFO: Gathering OVN_Southbound from ovnkube-node-rkx2v...
2026-06-05T11:54:44.149790593Z INFO: Gathering OVN_Southbound from ovnkube-node-vnn5f...
2026-06-05T11:54:44.198820732Z Wrote inspect data to must-gather.
2026-06-05T11:54:44.243788240Z INFO: Getting status/flags from prometheus-k8s-0
2026-06-05T11:54:44.346003135Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-06-05T11:54:44.471291479Z tar: Removing leading `/' from member names
2026-06-05T11:54:44.475976975Z tar: Removing leading `/' from member names
2026-06-05T11:54:44.502798839Z tar: Removing leading `/' from member names
2026-06-05T11:54:44.527680066Z tar: Removing leading `/' from member names
2026-06-05T11:54:44.556163894Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-06-05T11:54:44.566161086Z tar: Removing leading `/' from member names
2026-06-05T11:54:44.574918663Z tar: Removing leading `/' from member names
2026-06-05T11:54:44.609278646Z Wrote inspect data to must-gather.
2026-06-05T11:54:44.618918605Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:44.638303609Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-06-05T11:54:44.860139998Z Wrote inspect data to must-gather.
2026-06-05T11:54:44.883914444Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:44.888367949Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-06-05T11:54:44.910423602Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-06-05T11:54:45.017364381Z INFO: Waiting for network log collection to complete ...
2026-06-05T11:54:45.017401197Z INFO: Waiting for ovnk database copies to complete ...
2026-06-05T11:54:45.017896872Z INFO: Copying ovnk databases complete.
2026-06-05T11:54:45.020315682Z 3.0M	must-gather/network_logs/ovnk_database_store
2026-06-05T11:54:45.023267599Z ovnk_database_store/
2026-06-05T11:54:45.023267599Z ovnk_database_store/ovnkube-node-rkx2v_nbdb
2026-06-05T11:54:45.034527840Z ovnk_database_store/ovnkube-node-vnn5f_sbdb
2026-06-05T11:54:45.053334587Z ovnk_database_store/ovnkube-node-rkx2v_sbdb
2026-06-05T11:54:45.070559495Z ovnk_database_store/ovnkube-node-vnn5f_nbdb
2026-06-05T11:54:45.079261620Z ovnk_database_store/ovnkube-node-5ctg8_sbdb
2026-06-05T11:54:45.095959543Z W0605 11:54:45.095910    1477 util.go:195] skipping , failed to read event err: Object 'Kind' is missing in ''
2026-06-05T11:54:45.099690173Z ovnk_database_store/ovnkube-node-5ctg8_nbdb
2026-06-05T11:54:45.101396853Z Wrote inspect data to must-gather.
2026-06-05T11:54:45.103656389Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:45.111548796Z INFO: Network log collection complete.
2026-06-05T11:54:45.165218891Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-06-05T11:54:45.222826692Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-06-05T11:54:45.323809465Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:45.326999150Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-06-05T11:54:45.400123530Z Wrote inspect data to must-gather.
2026-06-05T11:54:45.407324371Z Gathering data for ns/openshift-network-console...
2026-06-05T11:54:45.549673044Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:45.567642481Z INFO: Getting status from alertmanager-main-0
2026-06-05T11:54:45.706787819Z Wrote inspect data to must-gather.
2026-06-05T11:54:45.759624048Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:45.763086829Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-06-05T11:54:45.897884317Z Gathering data for ns/openshift-console-operator...
2026-06-05T11:54:45.975573035Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-06-05T11:54:45.978461153Z INFO: Fallback to identify the container image from release info
2026-06-05T11:54:45.998901746Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:46.182509116Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:46.185351591Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-06-05T11:54:46.303082489Z Gathering data for ns/openshift-console...
2026-06-05T11:54:46.349888959Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:46.538229045Z Wrote inspect data to /must-gather/istio.
2026-06-05T11:54:46.769429885Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:75a8fe9618bb81a42d565c535f20bf55d78662baa7f32548859596b4c5aa793e
2026-06-05T11:54:46.854443371Z Gathering data for ns/openshift-cluster-storage-operator...
2026-06-05T11:54:46.895877965Z daemonset.apps/perf-node-gather-daemonset created
2026-06-05T11:54:47.024132470Z Waiting for performance profile collector pods to become ready: 1
2026-06-05T11:54:47.238658711Z Done executing Istio gather script
2026-06-05T11:54:47.413558217Z Gathering data for ns/openshift-dns-operator...
2026-06-05T11:54:47.692333849Z Gathering data for ns/openshift-dns...
2026-06-05T11:54:47.873364886Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-05T11:54:48.138260047Z Waiting for performance profile collector pods to become ready: 2
2026-06-05T11:54:48.317213459Z Gathering data for ns/openshift-image-registry...
2026-06-05T11:54:48.776813448Z Gathering data for ns/openshift-ingress-operator...
2026-06-05T11:54:49.085040038Z Gathering data for ns/openshift-ingress...
2026-06-05T11:54:49.250042795Z Waiting for performance profile collector pods to become ready: 3
2026-06-05T11:54:49.412865141Z Gathering data for ns/openshift-ingress-canary...
2026-06-05T11:54:49.806162370Z Gathering data for ns/openshift-insights...
2026-06-05T11:54:50.359045055Z Waiting for performance profile collector pods to become ready: 4
2026-06-05T11:54:51.476611178Z Waiting for performance profile collector pods to become ready: 5
2026-06-05T11:54:51.760738080Z Gathering data for ns/openshift-jobset-operator...
2026-06-05T11:54:52.588418531Z Gathering data for ns/openshift-config...
2026-06-05T11:54:52.597694215Z Waiting for performance profile collector pods to become ready: 6
2026-06-05T11:54:52.858137302Z Gathering data for ns/openshift-config-managed...
2026-06-05T11:54:52.879317690Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-05T11:54:53.324025131Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-06-05T11:54:53.631600534Z Gathering data for ns/openshift-kube-apiserver...
2026-06-05T11:54:53.727885655Z Waiting for performance profile collector pods to become ready: 7
2026-06-05T11:54:53.907408663Z Gathering data for ns/openshift-kube-controller-manager...
2026-06-05T11:54:54.208263693Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-06-05T11:54:54.512300434Z Gathering data for ns/openshift-kube-scheduler...
2026-06-05T11:54:54.785011249Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-06-05T11:54:54.857194650Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-06-05T11:54:55.101003289Z Collecting performance related data for node ip-10-0-130-33.ec2.internal
2026-06-05T11:54:55.104955793Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-06-05T11:54:55.108312657Z Collecting performance related data for node ip-10-0-138-129.ec2.internal
2026-06-05T11:54:55.116075889Z Collecting performance related data for node ip-10-0-130-222.ec2.internal
2026-06-05T11:54:55.518106233Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-06-05T11:54:55.880972643Z Gathering data for ns/openshift-user-workload-monitoring...
2026-06-05T11:54:56.510126844Z Gathering data for ns/openshift-multus...
2026-06-05T11:54:57.885523186Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-05T11:54:58.019122583Z Gathering data for ns/openshift-ovn-kubernetes...
2026-06-05T11:54:59.301857402Z Gathering data for ns/openshift-host-network...
2026-06-05T11:54:59.661023983Z Gathering data for ns/openshift-network-diagnostics...
2026-06-05T11:55:00.219908275Z Gathering data for ns/openshift-network-node-identity...
2026-06-05T11:55:00.661117305Z Gathering data for ns/openshift-network-operator...
2026-06-05T11:55:01.049583491Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-06-05T11:55:01.361850586Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-06-05T11:55:01.860430414Z Gathering data for ns/openshift-apiserver-operator...
2026-06-05T11:55:02.203620579Z Gathering data for ns/openshift-apiserver...
2026-06-05T11:55:02.658014730Z Gathering data for ns/openshift-controller-manager-operator...
2026-06-05T11:55:02.893600641Z [disk usage checker] Volume usage percentage: current = 18 ; allowed = 70
2026-06-05T11:55:02.933181220Z Gathering data for ns/openshift-controller-manager...
2026-06-05T11:55:02.951236531Z Collecting kubelet logs for node ip-10-0-130-33.ec2.internal
2026-06-05T11:55:02.953508733Z Collecting kubelet logs for node ip-10-0-138-129.ec2.internal
2026-06-05T11:55:02.955706303Z Collecting kubelet logs for node ip-10-0-130-222.ec2.internal
2026-06-05T11:55:03.261349103Z Gathering data for ns/openshift-cluster-samples-operator...
2026-06-05T11:55:03.312877710Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-pxntj namespace
2026-06-05T11:55:03.317845843Z INFO: Node performance data collection complete.
2026-06-05T11:55:03.954803622Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-06-05T11:55:04.237221555Z Gathering data for ns/openshift-service-ca-operator...
2026-06-05T11:55:04.559105150Z Gathering data for ns/openshift-service-ca...
2026-06-05T11:55:04.905080275Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-06-05T11:55:05.791209860Z Wrote inspect data to must-gather.
2026-06-05T11:55:05.791257503Z error: inspection completed with the errors occurred while gathering data:
2026-06-05T11:55:05.791257503Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-06-05T11:55:05.916689394Z error: the server doesn't have a resource type "clusters"
2026-06-05T11:55:06.042918717Z Caches written to disk