2026-10-30T14:22:05.263739435Z [disk usage checker] Started
2026-10-30T14:22:05.267024795Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:05.653127440Z Gathering data for ns/openshift-cluster-version...
2026-10-30T14:22:05.945845800Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-10-30T14:22:06.203851160Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-10-30T14:22:06.406493828Z Gathering data for ns/default...
2026-10-30T14:22:07.124466275Z Gathering data for ns/openshift...
2026-10-30T14:22:07.956856474Z Gathering data for ns/kube-system...
2026-10-30T14:22:08.838701900Z Waiting on subprocesses to finish execution.
2026-10-30T14:22:08.844615496Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-10-30T14:22:08.847396026Z INFO: Gathering HAProxy config files
2026-10-30T14:22:08.847809451Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-10-30T14:22:08.850142353Z Executing Istio gather script
2026-10-30T14:22:08.858295830Z INFO: Collecting host service logs for crio
2026-10-30T14:22:08.858628707Z INFO: Collecting host service logs for kubelet
2026-10-30T14:22:08.858993478Z INFO: Collecting host service logs for rpm-ostreed
2026-10-30T14:22:08.859366893Z INFO: Collecting host service logs for ostree-finalize-staged
2026-10-30T14:22:08.859716262Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-10-30T14:22:08.860078689Z INFO: Collecting host service logs for machine-config-daemon-host
2026-10-30T14:22:08.860536716Z INFO: Collecting host service logs for NetworkManager
2026-10-30T14:22:08.860935678Z INFO: Collecting host service logs for openvswitch
2026-10-30T14:22:08.861407755Z INFO: Collecting host service logs for ovs-configuration
2026-10-30T14:22:08.861769916Z INFO: Collecting host service logs for ovsdb-server
2026-10-30T14:22:08.862113654Z INFO: Collecting host service logs for ovs-vswitchd
2026-10-30T14:22:08.862562572Z INFO: Waiting for worker host service log collection to complete ...
2026-10-30T14:22:08.866721585Z INFO: Gathering machine config daemon's old logs from all nodes
2026-10-30T14:22:08.934875896Z INFO: Waiting for node performance related collection to complete ...
2026-10-30T14:22:09.653730957Z INFO: "metallb-operator" not detected. Skipping.
2026-10-30T14:22:09.756379088Z INFO: "nfd" not detected. Skipping.
2026-10-30T14:22:09.808895336Z No resources found in openshift-etcd namespace.
2026-10-30T14:22:09.816767681Z Wrote inspect data to must-gather.
2026-10-30T14:22:09.924556874Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-10-30T14:22:10.005840002Z INFO: "sriov-network-operator" not detected. Skipping.
2026-10-30T14:22:10.027374573Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-10-30T14:22:10.027403471Z INFO: on-disk MachineConfig config collection complete.
2026-10-30T14:22:10.139459367Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-10-30T14:22:10.158531043Z INFO: Found 1 replicas - prometheus-k8s-0
2026-10-30T14:22:10.169302400Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-8nhcv
2026-10-30T14:22:10.169302400Z insights-runtime-extractor-57jr4
2026-10-30T14:22:10.169302400Z insights-runtime-extractor-5mgn9
2026-10-30T14:22:10.169302400Z insights-runtime-extractor-x8txr
2026-10-30T14:22:10.208607814Z No resources found
2026-10-30T14:22:10.316956021Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:10.385177589Z error: the server doesn't have a resource type "performanceprofile"
2026-10-30T14:22:10.626734208Z ERROR: No running kube-apiserver pods found
2026-10-30T14:22:10.633488113Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-10-30T14:22:10.633488113Z See 'oc rsync -h' for help and examples
2026-10-30T14:22:10.885935505Z Wrote inspect data to must-gather.
2026-10-30T14:22:10.888019613Z INFO: Worker host service log collection to complete.
2026-10-30T14:22:11.081302636Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-10-30T14:22:11.142934920Z Wrote inspect data to must-gather.
2026-10-30T14:22:11.204748408Z Wrote inspect data to must-gather.
2026-10-30T14:22:11.281996715Z Wrote inspect data to must-gather.
2026-10-30T14:22:11.334883426Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-10-30T14:22:11.362206313Z No resources found
2026-10-30T14:22:11.421924597Z INFO: 'previous-logs' folder not found on ip-10-0-128-234.ec2.internal, skipping...
2026-10-30T14:22:11.456526204Z Wrote inspect data to must-gather.
2026-10-30T14:22:11.461589356Z Wrote inspect data to must-gather.
2026-10-30T14:22:11.480218803Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-10-30T14:22:11.520030928Z INFO: Waiting for HAProxy config collection to complete ...
2026-10-30T14:22:11.668194005Z Gathering data for ns/openshift-cloud-credential-operator...
2026-10-30T14:22:11.785309354Z Wrote inspect data to must-gather.
2026-10-30T14:22:11.821332406Z error: the server doesn't have a resource type "machineconfigs"
2026-10-30T14:22:11.855906465Z INFO: 'previous-logs' folder not found on ip-10-0-132-70.ec2.internal, skipping...
2026-10-30T14:22:11.937412324Z tar: Removing leading `/' from member names
2026-10-30T14:22:11.954538673Z INFO: HAProxy config collection complete.
2026-10-30T14:22:12.054808482Z Gathering data for ns/openshift-etcd...
2026-10-30T14:22:12.108039891Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+
2026-10-30T14:22:12.120033687Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-10-30T14:22:12.151396855Z error: the server doesn't have a resource type "machineconfigpools"
2026-10-30T14:22:12.155214646Z INFO: Getting rules from prometheus-k8s-0
2026-10-30T14:22:12.254631213Z INFO: 'previous-logs' folder not found on ip-10-0-134-107.ec2.internal, skipping...
2026-10-30T14:22:12.254674137Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-10-30T14:22:12.254674137Z INFO: Machine Config Daemon termination log collection complete.
2026-10-30T14:22:12.274239012Z Wrote inspect data to must-gather.
2026-10-30T14:22:12.430450509Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-10-30T14:22:12.492563222Z Wrote inspect data to must-gather.
2026-10-30T14:22:12.709969167Z Wrote inspect data to must-gather.
2026-10-30T14:22:12.754121252Z error: the server doesn't have a resource type "kubeletconfigs"
2026-10-30T14:22:12.956033665Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io
2026-10-30T14:22:12.983297177Z INFO: Getting status/config from prometheus-k8s-0
2026-10-30T14:22:13.070290713Z Wrote inspect data to must-gather.
2026-10-30T14:22:13.070326498Z error: inspection completed with the errors occurred while gathering data:
2026-10-30T14:22:13.070326498Z     namespaces "assisted-installer" not found
2026-10-30T14:22:13.136697676Z Wrote inspect data to must-gather.
2026-10-30T14:22:13.195764975Z INFO: LEGACY MODE
2026-10-30T14:22:13.195806958Z INFO: Gathering ovn-kubernetes master data
2026-10-30T14:22:13.269095297Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:13.290474461Z Gathering data for ns/openshift-monitoring...
2026-10-30T14:22:13.405432546Z Wrote inspect data to must-gather.
2026-10-30T14:22:13.617579757Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:13.621559919Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-10-30T14:22:13.680727601Z Wrote inspect data to must-gather.
2026-10-30T14:22:13.711783719Z INFO: Getting status/flags from prometheus-k8s-0
2026-10-30T14:22:13.878885011Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:13.961194675Z Wrote inspect data to must-gather.
2026-10-30T14:22:14.078017052Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-10-30T14:22:14.220997143Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:14.223761187Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-10-30T14:22:14.258028227Z Wrote inspect data to must-gather.
2026-10-30T14:22:14.264900007Z INFO: Waiting for network log collection to complete ...
2026-10-30T14:22:14.264948019Z INFO: Waiting for ovnk database copies to complete ...
2026-10-30T14:22:14.265464683Z INFO: Copying ovnk databases complete.
2026-10-30T14:22:14.267189258Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory
2026-10-30T14:22:14.267716356Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 686 is not a child of this shell
2026-10-30T14:22:14.267716356Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 688 is not a child of this shell
2026-10-30T14:22:14.267716356Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 689 is not a child of this shell
2026-10-30T14:22:14.267716356Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 690 is not a child of this shell
2026-10-30T14:22:14.267716356Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 691 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 692 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 693 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 694 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 703 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 704 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 705 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 706 is not a child of this shell
2026-10-30T14:22:14.267743812Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 707 is not a child of this shell
2026-10-30T14:22:14.270910762Z tar: ovnk_database_store: Cannot stat: No such file or directory
2026-10-30T14:22:14.274542276Z tar: Exiting with failure status due to previous errors
2026-10-30T14:22:14.275133299Z INFO: Network log collection complete.
2026-10-30T14:22:14.453188798Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-10-30T14:22:14.505692410Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:14.570204690Z Wrote inspect data to must-gather.
2026-10-30T14:22:14.792450432Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:14.795763431Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-10-30T14:22:14.852546790Z Wrote inspect data to must-gather.
2026-10-30T14:22:14.907897057Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-10-30T14:22:15.050897364Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:15.257677342Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod
2026-10-30T14:22:15.344044999Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:15.358382680Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:15.361712692Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-10-30T14:22:15.460316490Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c
2026-10-30T14:22:15.472423380Z INFO: Getting status from alertmanager-main-0
2026-10-30T14:22:15.631431379Z daemonset.apps/perf-node-gather-daemonset created
2026-10-30T14:22:15.642673372Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:15.800015342Z Waiting for performance profile collector pods to become ready: 1
2026-10-30T14:22:15.979551329Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:15.982030995Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-10-30T14:22:16.196156134Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:16.495306177Z Wrote inspect data to /must-gather/istio.
2026-10-30T14:22:16.958303769Z Waiting for performance profile collector pods to become ready: 2
2026-10-30T14:22:17.564820180Z Done executing Istio gather script
2026-10-30T14:22:18.099215937Z Waiting for performance profile collector pods to become ready: 3
2026-10-30T14:22:19.231692731Z Waiting for performance profile collector pods to become ready: 4
2026-10-30T14:22:20.349483139Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:20.372442497Z Waiting for performance profile collector pods to become ready: 5
2026-10-30T14:22:21.005581071Z Gathering data for ns/openshift-network-console...
2026-10-30T14:22:21.516930956Z Waiting for performance profile collector pods to become ready: 6
2026-10-30T14:22:22.052869518Z Gathering data for ns/openshift-console-operator...
2026-10-30T14:22:22.659020078Z Waiting for performance profile collector pods to become ready: 7
2026-10-30T14:22:23.030562615Z Gathering data for ns/openshift-console...
2026-10-30T14:22:23.810862042Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-10-30T14:22:24.158875189Z Collecting performance related data for node ip-10-0-132-70.ec2.internal
2026-10-30T14:22:24.168122805Z Collecting performance related data for node ip-10-0-128-234.ec2.internal
2026-10-30T14:22:24.176626889Z Collecting performance related data for node ip-10-0-134-107.ec2.internal
2026-10-30T14:22:24.339523762Z Gathering data for ns/openshift-cluster-storage-operator...
2026-10-30T14:22:25.357217953Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:25.683931880Z Gathering data for ns/openshift-dns-operator...
2026-10-30T14:22:26.644448204Z Gathering data for ns/openshift-dns...
2026-10-30T14:22:29.146617936Z Gathering data for ns/openshift-image-registry...
2026-10-30T14:22:30.361923749Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:30.545532271Z Gathering data for ns/openshift-ingress-operator...
2026-10-30T14:22:31.378216148Z Gathering data for ns/openshift-ingress...
2026-10-30T14:22:32.346535064Z Gathering data for ns/openshift-ingress-canary...
2026-10-30T14:22:32.708055115Z Collecting kubelet logs for node ip-10-0-132-70.ec2.internal
2026-10-30T14:22:32.709799159Z Collecting kubelet logs for node ip-10-0-128-234.ec2.internal
2026-10-30T14:22:32.711529677Z Collecting kubelet logs for node ip-10-0-134-107.ec2.internal
2026-10-30T14:22:33.216138962Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-b2j9p namespace
2026-10-30T14:22:33.234225544Z INFO: Node performance data collection complete.
2026-10-30T14:22:33.619656353Z Gathering data for ns/openshift-insights...
2026-10-30T14:22:35.366664816Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:37.361377204Z Gathering data for ns/openshift-jobset-operator...
2026-10-30T14:22:38.904746453Z Gathering data for ns/openshift-config...
2026-10-30T14:22:39.657070066Z Gathering data for ns/openshift-config-managed...
2026-10-30T14:22:40.370993097Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:40.520992302Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-10-30T14:22:41.244509575Z Gathering data for ns/openshift-kube-apiserver...
2026-10-30T14:22:41.996982438Z Gathering data for ns/openshift-kube-controller-manager...
2026-10-30T14:22:42.770332298Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-10-30T14:22:43.589643494Z Gathering data for ns/openshift-kube-scheduler...
2026-10-30T14:22:44.320292062Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-10-30T14:22:45.115509998Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-10-30T14:22:45.375564094Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:46.146293166Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-10-30T14:22:47.039081535Z Gathering data for ns/openshift-user-workload-monitoring...
2026-10-30T14:22:48.197741053Z Gathering data for ns/openshift-multus...
2026-10-30T14:22:50.379874155Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:53.931434312Z Gathering data for ns/openshift-ovn-kubernetes...
2026-10-30T14:22:55.384094671Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:22:58.651143750Z Gathering data for ns/openshift-host-network...
2026-10-30T14:22:59.524339032Z Gathering data for ns/openshift-network-diagnostics...
2026-10-30T14:23:00.388466221Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:23:01.262506257Z Gathering data for ns/openshift-network-node-identity...
2026-10-30T14:23:02.511511422Z Gathering data for ns/openshift-network-operator...
2026-10-30T14:23:03.768050270Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-10-30T14:23:04.522527904Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-10-30T14:23:05.392793154Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:23:05.921860100Z Gathering data for ns/openshift-apiserver-operator...
2026-10-30T14:23:06.705485950Z Gathering data for ns/openshift-apiserver...
2026-10-30T14:23:07.798772327Z Gathering data for ns/openshift-controller-manager-operator...
2026-10-30T14:23:08.527428106Z Gathering data for ns/openshift-controller-manager...
2026-10-30T14:23:09.278731227Z Gathering data for ns/openshift-cluster-samples-operator...
2026-10-30T14:23:10.397189962Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:23:10.678134278Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-10-30T14:23:11.457881107Z Gathering data for ns/openshift-service-ca-operator...
2026-10-30T14:23:12.341385113Z Gathering data for ns/openshift-service-ca...
2026-10-30T14:23:13.430624138Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-10-30T14:23:15.402198851Z [disk usage checker] Volume usage percentage: current = 9 ; allowed = 70
2026-10-30T14:23:15.938503825Z Wrote inspect data to must-gather.
2026-10-30T14:23:15.938553860Z error: inspection completed with the errors occurred while gathering data:
2026-10-30T14:23:15.938553860Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-10-30T14:23:16.142418434Z error: the server doesn't have a resource type "clusters"
2026-10-30T14:23:16.221403218Z Caches written to disk