2026-07-15T13:23:46.435983001Z [disk usage checker] Started
2026-07-15T13:23:46.439683713Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-15T13:23:46.627477769Z Gathering data for ns/openshift-cluster-version...
2026-07-15T13:23:46.771973483Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-15T13:23:46.867838375Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-15T13:23:46.946740540Z Gathering data for ns/default...
2026-07-15T13:23:47.283107323Z Gathering data for ns/openshift...
2026-07-15T13:23:47.716050106Z Gathering data for ns/kube-system...
2026-07-15T13:23:48.396245635Z Gathering data for ns/openshift-etcd...
2026-07-15T13:23:48.734276951Z Wrote inspect data to must-gather.
2026-07-15T13:23:48.734313527Z error: inspection completed with the errors occurred while gathering data:
2026-07-15T13:23:48.734313527Z     namespaces "assisted-installer" not found
2026-07-15T13:23:48.902813662Z Waiting on subprocesses to finish execution.
2026-07-15T13:23:48.908033413Z INFO: Gathering HAProxy config files
2026-07-15T13:23:48.911580834Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-15T13:23:48.911607799Z Executing Istio gather script
2026-07-15T13:23:48.920713474Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-15T13:23:48.928097673Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-15T13:23:48.938570103Z INFO: Collecting host service logs for crio
2026-07-15T13:23:48.938891397Z INFO: Collecting host service logs for kubelet
2026-07-15T13:23:48.939220702Z INFO: Collecting host service logs for rpm-ostreed
2026-07-15T13:23:48.939537168Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-15T13:23:48.939856462Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-15T13:23:48.940160627Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-15T13:23:48.940492856Z INFO: Collecting host service logs for NetworkManager
2026-07-15T13:23:48.940790570Z INFO: Collecting host service logs for openvswitch
2026-07-15T13:23:48.941089265Z INFO: Collecting host service logs for ovs-configuration
2026-07-15T13:23:48.943211512Z INFO: Collecting host service logs for ovsdb-server
2026-07-15T13:23:48.943211512Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-15T13:23:48.943211512Z INFO: Waiting for worker host service log collection to complete ...
2026-07-15T13:23:49.170581325Z INFO: Waiting for node performance related collection to complete ...
2026-07-15T13:23:49.442391722Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-15T13:23:49.504654593Z No resources found in openshift-etcd namespace.
2026-07-15T13:23:49.717313435Z INFO: "metallb-operator" not detected. Skipping.
2026-07-15T13:23:49.780152879Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-15T13:23:49.832471732Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-15T13:23:49.832555429Z INFO: on-disk MachineConfig config collection complete.
2026-07-15T13:23:49.845862695Z INFO: Collecting Insights Archives from insights-operator-68d9dd94cf-kmsxp
2026-07-15T13:23:49.845908002Z insights-runtime-extractor-5vb72
2026-07-15T13:23:49.845929558Z insights-runtime-extractor-6r5z5
2026-07-15T13:23:49.845949115Z insights-runtime-extractor-8jc26
2026-07-15T13:23:49.870603152Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-15T13:23:49.876009797Z error: the server doesn't have a resource type "performanceprofile"
2026-07-15T13:23:50.032040161Z No resources found
2026-07-15T13:23:50.156927643Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-15T13:23:50.234309633Z Wrote inspect data to must-gather.
2026-07-15T13:23:50.297390945Z INFO: Worker host service log collection to complete.
2026-07-15T13:23:50.305295800Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-15T13:23:50.305295800Z See 'oc rsync -h' for help and examples
2026-07-15T13:23:50.375051894Z Wrote inspect data to must-gather.
2026-07-15T13:23:50.393291672Z ERROR: No running kube-apiserver pods found
2026-07-15T13:23:50.529981232Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-15T13:23:50.588745781Z Wrote inspect data to must-gather.
2026-07-15T13:23:50.777804731Z No resources found
2026-07-15T13:23:50.893253094Z INFO: 'previous-logs' folder not found on ip-10-0-128-4.ec2.internal, skipping...
2026-07-15T13:23:51.070164393Z Wrote inspect data to must-gather.
2026-07-15T13:23:51.091725700Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-15T13:23:51.102638244Z Wrote inspect data to must-gather.
2026-07-15T13:23:51.146481085Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-15T13:23:51.249624042Z Gathering data for ns/openshift-monitoring...
2026-07-15T13:23:51.311653365Z error: the server doesn't have a resource type "machineconfigs"
2026-07-15T13:23:51.377623931Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-15T13:23:51.454304302Z Wrote inspect data to must-gather.
2026-07-15T13:23:51.463340497Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-15T13:23:51.502374226Z Wrote inspect data to must-gather.
2026-07-15T13:23:51.610119034Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-15T13:23:51.615599601Z tar: Removing leading `/' from member names
2026-07-15T13:23:51.620988293Z INFO: HAProxy config collection complete.
2026-07-15T13:23:51.629433381Z INFO: Getting rules from prometheus-k8s-0
2026-07-15T13:23:51.644569270Z INFO: 'previous-logs' folder not found on ip-10-0-129-104.ec2.internal, skipping...
2026-07-15T13:23:51.803902942Z Wrote inspect data to must-gather.
2026-07-15T13:23:51.885800121Z Wrote inspect data to must-gather.
2026-07-15T13:23:51.904021450Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-15T13:23:51.929515170Z INFO: 'previous-logs' folder not found on ip-10-0-138-25.ec2.internal, skipping...
2026-07-15T13:23:51.929627547Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-15T13:23:51.929687335Z INFO: Machine Config Daemon termination log collection complete.
2026-07-15T13:23:52.054711439Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-15T13:23:52.065203044Z Wrote inspect data to must-gather.
2026-07-15T13:23:52.066950067Z INFO: Getting status/config from prometheus-k8s-0
2026-07-15T13:23:52.277484892Z Wrote inspect data to must-gather.
2026-07-15T13:23:52.298133073Z Wrote inspect data to must-gather.
2026-07-15T13:23:52.463158203Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-15T13:23:52.481336318Z Wrote inspect data to must-gather.
2026-07-15T13:23:52.529609064Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-15T13:23:52.609291803Z INFO: INTERCONNECT MODE
2026-07-15T13:23:52.609388295Z INFO: Gathering ovn-kubernetes DBs
2026-07-15T13:23:52.694872837Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-15T13:23:52.724114635Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-15T13:23:52.729700988Z Wrote inspect data to must-gather.
2026-07-15T13:23:52.743798925Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:52.769747966Z INFO: Gathering OVN_Northbound from ovnkube-node-ms7vj...
2026-07-15T13:23:52.770093785Z INFO: Gathering OVN_Northbound from ovnkube-node-xj9vl...
2026-07-15T13:23:52.770420683Z INFO: Gathering OVN_Northbound from ovnkube-node-xx99p...
2026-07-15T13:23:52.770759523Z INFO: Gathering OVN_Southbound from ovnkube-node-ms7vj...
2026-07-15T13:23:52.771072527Z INFO: Gathering OVN_Southbound from ovnkube-node-xj9vl...
2026-07-15T13:23:52.771434676Z INFO: Gathering OVN_Southbound from ovnkube-node-xx99p...
2026-07-15T13:23:53.060215324Z Wrote inspect data to must-gather.
2026-07-15T13:23:53.064899437Z tar: Removing leading `/' from member names
2026-07-15T13:23:53.088563520Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-15T13:23:53.120709759Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:53.126069072Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-15T13:23:53.138812071Z tar: Removing leading `/' from member names
2026-07-15T13:23:53.170815006Z tar: Removing leading `/' from member names
2026-07-15T13:23:53.186972051Z tar: Removing leading `/' from member names
2026-07-15T13:23:53.204415836Z tar: Removing leading `/' from member names
2026-07-15T13:23:53.206982151Z tar: Removing leading `/' from member names
2026-07-15T13:23:53.345652271Z Wrote inspect data to must-gather.
2026-07-15T13:23:53.357716080Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:53.363967044Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-15T13:23:53.570520011Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-15T13:23:53.607727825Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:53.612778700Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-15T13:23:53.673897827Z INFO: Waiting for network log collection to complete ...
2026-07-15T13:23:53.674018963Z INFO: Waiting for ovnk database copies to complete ...
2026-07-15T13:23:53.674569327Z INFO: Copying ovnk databases complete.
2026-07-15T13:23:53.676330460Z 3.0M	must-gather/network_logs/ovnk_database_store
2026-07-15T13:23:53.678567900Z ovnk_database_store/
2026-07-15T13:23:53.678567900Z ovnk_database_store/ovnkube-node-ms7vj_nbdb
2026-07-15T13:23:53.688364780Z ovnk_database_store/ovnkube-node-xj9vl_nbdb
2026-07-15T13:23:53.695597399Z ovnk_database_store/ovnkube-node-xj9vl_sbdb
2026-07-15T13:23:53.716485142Z ovnk_database_store/ovnkube-node-ms7vj_sbdb
2026-07-15T13:23:53.731576715Z Wrote inspect data to must-gather.
2026-07-15T13:23:53.740884895Z ovnk_database_store/ovnkube-node-xx99p_sbdb
2026-07-15T13:23:53.760029136Z ovnk_database_store/ovnkube-node-xx99p_nbdb
2026-07-15T13:23:53.772117644Z INFO: Network log collection complete.
2026-07-15T13:23:53.774499540Z Gathering data for ns/openshift-network-console...
2026-07-15T13:23:53.774589751Z INFO: Getting status from alertmanager-main-0
2026-07-15T13:23:53.834641824Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:54.041291702Z Wrote inspect data to must-gather.
2026-07-15T13:23:54.051103187Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:54.052983993Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-15T13:23:54.229915770Z Gathering data for ns/openshift-console-operator...
2026-07-15T13:23:54.276322474Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:54.296223184Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-15T13:23:54.298517464Z INFO: Fallback to identify the container image from release info
2026-07-15T13:23:54.473330993Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:54.477162992Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-15T13:23:54.626648331Z Gathering data for ns/openshift-console...
2026-07-15T13:23:54.636643752Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:54.809611431Z Wrote inspect data to /must-gather/istio.
2026-07-15T13:23:55.099784258Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d8d5dc4ea635b13544e6f92103089219e732369ae30557b7beee2e8a11bf500d
2026-07-15T13:23:55.212439098Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-15T13:23:55.227892976Z daemonset.apps/perf-node-gather-daemonset created
2026-07-15T13:23:55.371354006Z Waiting for performance profile collector pods to become ready: 1
2026-07-15T13:23:55.471812451Z Done executing Istio gather script
2026-07-15T13:23:55.751068543Z Gathering data for ns/openshift-dns-operator...
2026-07-15T13:23:56.064651958Z Gathering data for ns/openshift-dns...
2026-07-15T13:23:56.469251522Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-15T13:23:56.480695444Z Waiting for performance profile collector pods to become ready: 2
2026-07-15T13:23:56.639075007Z Gathering data for ns/openshift-image-registry...
2026-07-15T13:23:57.061726408Z Gathering data for ns/openshift-ingress-operator...
2026-07-15T13:23:57.413881662Z Gathering data for ns/openshift-ingress...
2026-07-15T13:23:57.579517079Z Waiting for performance profile collector pods to become ready: 3
2026-07-15T13:23:57.730902349Z Gathering data for ns/openshift-ingress-canary...
2026-07-15T13:23:58.107552018Z Gathering data for ns/openshift-insights...
2026-07-15T13:23:58.697110899Z Waiting for performance profile collector pods to become ready: 4
2026-07-15T13:23:59.798826544Z Waiting for performance profile collector pods to become ready: 5
2026-07-15T13:23:59.908359505Z Gathering data for ns/openshift-jobset-operator...
2026-07-15T13:24:00.685380076Z Gathering data for ns/openshift-config...
2026-07-15T13:24:00.903825243Z Waiting for performance profile collector pods to become ready: 6
2026-07-15T13:24:00.994823454Z Gathering data for ns/openshift-config-managed...
2026-07-15T13:24:01.402554352Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-15T13:24:01.474076474Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-15T13:24:01.656515226Z Gathering data for ns/openshift-kube-apiserver...
2026-07-15T13:24:01.944157961Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-15T13:24:02.028425112Z Waiting for performance profile collector pods to become ready: 7
2026-07-15T13:24:02.249727543Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-15T13:24:02.521700893Z Gathering data for ns/openshift-kube-scheduler...
2026-07-15T13:24:02.768975424Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-15T13:24:03.068816328Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-15T13:24:03.139811935Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-15T13:24:03.441447673Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-15T13:24:03.448053234Z Collecting performance related data for node ip-10-0-138-25.ec2.internal
2026-07-15T13:24:03.454814795Z Collecting performance related data for node ip-10-0-128-4.ec2.internal
2026-07-15T13:24:03.462024540Z Collecting performance related data for node ip-10-0-129-104.ec2.internal
2026-07-15T13:24:03.824186062Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-15T13:24:04.433654892Z Gathering data for ns/openshift-multus...
2026-07-15T13:24:06.065789786Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-15T13:24:06.478746260Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-15T13:24:07.324147330Z Gathering data for ns/openshift-host-network...
2026-07-15T13:24:07.630513005Z Gathering data for ns/openshift-network-diagnostics...
2026-07-15T13:24:08.155565199Z Gathering data for ns/openshift-network-node-identity...
2026-07-15T13:24:08.555891893Z Gathering data for ns/openshift-network-operator...
2026-07-15T13:24:08.913958580Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-15T13:24:09.188537401Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-15T13:24:09.669043440Z Gathering data for ns/openshift-apiserver-operator...
2026-07-15T13:24:09.943495485Z Gathering data for ns/openshift-apiserver...
2026-07-15T13:24:10.312487050Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-15T13:24:10.601807638Z Gathering data for ns/openshift-controller-manager...
2026-07-15T13:24:10.867995081Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-15T13:24:11.324046668Z Collecting kubelet logs for node ip-10-0-138-25.ec2.internal
2026-07-15T13:24:11.325826922Z Collecting kubelet logs for node ip-10-0-128-4.ec2.internal
2026-07-15T13:24:11.327911212Z Collecting kubelet logs for node ip-10-0-129-104.ec2.internal
2026-07-15T13:24:11.483378558Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-15T13:24:11.559350867Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-15T13:24:11.661571388Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-z8qcw namespace
2026-07-15T13:24:11.666385948Z INFO: Node performance data collection complete.
2026-07-15T13:24:11.842868778Z Gathering data for ns/openshift-service-ca-operator...
2026-07-15T13:24:12.146067906Z Gathering data for ns/openshift-service-ca...
2026-07-15T13:24:12.480901800Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-15T13:24:13.368822171Z Wrote inspect data to must-gather.
2026-07-15T13:24:13.368870080Z error: inspection completed with the errors occurred while gathering data:
2026-07-15T13:24:13.368870080Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-15T13:24:13.492651406Z error: the server doesn't have a resource type "clusters"
2026-07-15T13:24:13.629837345Z Caches written to disk