2026-10-31T07:57:30.388714599Z [disk usage checker] Started 2026-10-31T07:57:30.392411012Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:57:30.768220026Z Gathering data for ns/openshift-cluster-version... 2026-10-31T07:57:31.111997584Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-31T07:57:31.323433230Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-31T07:57:31.525452852Z Gathering data for ns/default... 2026-10-31T07:57:32.252077226Z Gathering data for ns/openshift... 2026-10-31T07:57:33.059683655Z Gathering data for ns/kube-system... 2026-10-31T07:57:34.658485152Z Waiting on subprocesses to finish execution. 2026-10-31T07:57:34.668154520Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time. 2026-10-31T07:57:34.673937233Z INFO: Gathering on-disk MachineConfig from degraded nodes 2026-10-31T07:57:34.676967174Z INFO: Gathering HAProxy config files 2026-10-31T07:57:34.684405264Z Executing Istio gather script 2026-10-31T07:57:34.694215495Z INFO: Collecting host service logs for crio 2026-10-31T07:57:34.694603739Z INFO: Collecting host service logs for kubelet 2026-10-31T07:57:34.694972853Z INFO: Collecting host service logs for rpm-ostreed 2026-10-31T07:57:34.695350754Z INFO: Collecting host service logs for ostree-finalize-staged 2026-10-31T07:57:34.695695676Z INFO: Collecting host service logs for machine-config-daemon-firstboot 2026-10-31T07:57:34.696038958Z INFO: Collecting host service logs for machine-config-daemon-host 2026-10-31T07:57:34.696410090Z INFO: Collecting host service logs for NetworkManager 2026-10-31T07:57:34.696765501Z INFO: Collecting host service logs for openvswitch 2026-10-31T07:57:34.697107332Z INFO: Collecting host service logs for ovs-configuration 2026-10-31T07:57:34.697467212Z INFO: Collecting host service logs for ovsdb-server 2026-10-31T07:57:34.697784243Z INFO: Collecting host service logs for ovs-vswitchd 2026-10-31T07:57:34.698162789Z INFO: Waiting for worker host service log collection to complete ... 2026-10-31T07:57:34.704661017Z INFO: Gathering machine config daemon's old logs from all nodes 2026-10-31T07:57:34.938156805Z INFO: Waiting for node performance related collection to complete ... 2026-10-31T07:57:35.338636013Z No resources found in openshift-etcd namespace. 2026-10-31T07:57:35.542359167Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:57:35.752475294Z INFO: Waiting for on-disk MachineConfig collection to complete ... 2026-10-31T07:57:35.752509874Z INFO: on-disk MachineConfig config collection complete. 2026-10-31T07:57:35.886134400Z INFO: Collecting Insights Archives from insights-operator-7c4986d449-vvm6c 2026-10-31T07:57:35.890032491Z insights-runtime-extractor-wqkwf 2026-10-31T07:57:35.890134798Z insights-runtime-extractor-zdg8b 2026-10-31T07:57:35.890161619Z insights-runtime-extractor-zpjzt 2026-10-31T07:57:35.900643840Z INFO: Found 1 replicas - prometheus-k8s-0 2026-10-31T07:57:35.949665220Z Wrote inspect data to must-gather. 2026-10-31T07:57:36.017834464Z No resources found 2026-10-31T07:57:36.126335409Z INFO: "metallb-operator" not detected. Skipping. 2026-10-31T07:57:36.127719934Z INFO: "nfd" not detected. Skipping. 2026-10-31T07:57:36.220962433Z INFO: "sriov-network-operator" not detected. Skipping. 2026-10-31T07:57:36.336010741Z INFO: namespace openshift-frr-k8s not detected. Skipping. 2026-10-31T07:57:36.504202106Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments 2026-10-31T07:57:36.504202106Z See 'oc rsync -h' for help and examples 2026-10-31T07:57:36.528509826Z INFO: Worker host service log collection to complete. 2026-10-31T07:57:36.531887542Z INFO: "kubernetes-nmstate-operator" not detected. Skipping. 2026-10-31T07:57:36.721382424Z INFO: Getting alertmanagers from prometheus-k8s-0 2026-10-31T07:57:36.875975640Z error: the server doesn't have a resource type "performanceprofile" 2026-10-31T07:57:37.046646042Z ERROR: No running kube-apiserver pods found 2026-10-31T07:57:37.254342289Z Gathering data for ns/openshift-etcd... 2026-10-31T07:57:37.255333233Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-31T07:57:37.293933790Z INFO: Waiting for HAProxy config collection to complete ... 2026-10-31T07:57:37.461759788Z No resources found 2026-10-31T07:57:37.475666906Z error: the server doesn't have a resource type "multi-networkpolicy" 2026-10-31T07:57:37.543539534Z Wrote inspect data to must-gather. 2026-10-31T07:57:37.564275357Z Wrote inspect data to must-gather. 2026-10-31T07:57:37.686752203Z Wrote inspect data to must-gather. 2026-10-31T07:57:37.720389279Z INFO: 'previous-logs' folder not found on ip-10-0-138-234.ec2.internal, skipping... 2026-10-31T07:57:37.778962881Z Wrote inspect data to must-gather. 2026-10-31T07:57:37.782773761Z Wrote inspect data to must-gather. 2026-10-31T07:57:37.787124562Z Wrote inspect data to must-gather. 2026-10-31T07:57:37.981885112Z tar: Removing leading `/' from member names 2026-10-31T07:57:38.001805687Z INFO: HAProxy config collection complete. 2026-10-31T07:57:38.048527228Z error: the server doesn't have a resource type "machineconfigs" 2026-10-31T07:57:38.113618736Z INFO: Getting rules from prometheus-k8s-0 2026-10-31T07:57:38.141844978Z INFO: 'previous-logs' folder not found on ip-10-0-138-89.ec2.internal, skipping... 2026-10-31T07:57:38.231965590Z Gathering data for ns/openshift-cloud-credential-operator... 2026-10-31T07:57:38.233266837Z Wrote inspect data to must-gather. 2026-10-31T07:57:38.372981301Z error: the server doesn't have a resource type "machineconfigpools" 2026-10-31T07:57:38.497118948Z Wrote inspect data to must-gather. 2026-10-31T07:57:38.497197608Z error: inspection completed with the errors occurred while gathering data: 2026-10-31T07:57:38.497197608Z namespaces "assisted-installer" not found 2026-10-31T07:57:38.531020975Z INFO: 'previous-logs' folder not found on ip-10-0-140-6.ec2.internal, skipping... 2026-10-31T07:57:38.531050026Z INFO: Waiting for Machine Config Daemon termination log collection to complete ... 2026-10-31T07:57:38.531068163Z INFO: Machine Config Daemon termination log collection complete. 2026-10-31T07:57:38.610586316Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable no sooner than v6.0+ 2026-10-31T07:57:38.685359553Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection. 2026-10-31T07:57:38.746842903Z Wrote inspect data to must-gather. 2026-10-31T07:57:38.757732678Z Wrote inspect data to must-gather. 2026-10-31T07:57:38.894731471Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2026-10-31T07:57:38.909132215Z INFO: Getting status/config from prometheus-k8s-0 2026-10-31T07:57:39.029125972Z error: the server doesn't have a resource type "kubeletconfigs" 2026-10-31T07:57:39.190432131Z Wrote inspect data to must-gather. 2026-10-31T07:57:39.409576087Z Wrote inspect data to must-gather. 2026-10-31T07:57:39.611544004Z INFO: Getting status/flags from prometheus-k8s-0 2026-10-31T07:57:39.651525415Z Gathering data for ns/openshift-monitoring... 2026-10-31T07:57:39.702504490Z INFO: LEGACY MODE 2026-10-31T07:57:39.702546826Z INFO: Gathering ovn-kubernetes master data 2026-10-31T07:57:39.804807512Z Wrote inspect data to must-gather. 2026-10-31T07:57:39.991280878Z INFO: Getting status/runtimeinfo from prometheus-k8s-0 2026-10-31T07:57:40.003499100Z Inspecting resource crd/backendtlspolicies.gateway.networking.k8s.io 2026-10-31T07:57:40.158727096Z Wrote inspect data to must-gather. 2026-10-31T07:57:40.317828362Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:40.368515289Z INFO: Getting targets?state=active from prometheus-k8s-0 2026-10-31T07:57:40.529618921Z Wrote inspect data to must-gather. 2026-10-31T07:57:40.681046160Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:40.683161528Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:57:40.684024616Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io 2026-10-31T07:57:40.855873853Z INFO: Getting status/tsdb from prometheus-k8s-0 2026-10-31T07:57:40.887286087Z Wrote inspect data to must-gather. 2026-10-31T07:57:40.989125202Z INFO: Waiting for network log collection to complete ... 2026-10-31T07:57:40.989269970Z INFO: Waiting for ovnk database copies to complete ... 2026-10-31T07:57:40.989767629Z INFO: Copying ovnk databases complete. 2026-10-31T07:57:40.991602580Z du: cannot access 'must-gather/network_logs/ovnk_database_store': No such file or directory 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 746 is not a child of this shell 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 747 is not a child of this shell 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 748 is not a child of this shell 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 749 is not a child of this shell 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 750 is not a child of this shell 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 751 is not a child of this shell 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 752 is not a child of this shell 2026-10-31T07:57:40.992145825Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 753 is not a child of this shell 2026-10-31T07:57:40.992174767Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 765 is not a child of this shell 2026-10-31T07:57:40.992174767Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 767 is not a child of this shell 2026-10-31T07:57:40.992174767Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 768 is not a child of this shell 2026-10-31T07:57:40.992174767Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 770 is not a child of this shell 2026-10-31T07:57:40.992174767Z /usr/bin/gather_network_logs_basics: line 190: wait: pid 771 is not a child of this shell 2026-10-31T07:57:40.995439545Z tar: ovnk_database_store: Cannot stat: No such file or directory 2026-10-31T07:57:40.998997479Z tar: Exiting with failure status due to previous errors 2026-10-31T07:57:40.999553600Z INFO: Network log collection complete. 2026-10-31T07:57:41.023474738Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:41.254629580Z Wrote inspect data to must-gather. 2026-10-31T07:57:41.380700114Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:41.383357444Z Inspecting resource crd/gateways.gateway.networking.k8s.io 2026-10-31T07:57:41.488624318Z INFO: Getting status from alertmanager-main-0 2026-10-31T07:57:41.596684547Z Wrote inspect data to must-gather. 2026-10-31T07:57:41.710031979Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:42.013244403Z INFO: Hosted cluster detected, get NTO container image from the TuneD pod 2026-10-31T07:57:42.053993059Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:42.056997740Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io 2026-10-31T07:57:42.249737238Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36f60e56e0594610b09d7a64639223d0a6a314315bca9313914f95c902a1597c 2026-10-31T07:57:42.360433053Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:42.487574388Z daemonset.apps/perf-node-gather-daemonset created 2026-10-31T07:57:42.681753150Z Waiting for performance profile collector pods to become ready: 1 2026-10-31T07:57:42.702154447Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:42.705294253Z Inspecting resource crd/httproutes.gateway.networking.k8s.io 2026-10-31T07:57:43.007821202Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:43.323034080Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:43.326019002Z Inspecting resource crd/inferencepools.inference.networking.k8s.io 2026-10-31T07:57:43.587242839Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:43.857513128Z Waiting for performance profile collector pods to become ready: 2 2026-10-31T07:57:43.925164872Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:43.927945757Z Inspecting resource crd/inferencepools.inference.networking.x-k8s.io 2026-10-31T07:57:44.188036573Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:44.515750875Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:44.518569084Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io 2026-10-31T07:57:44.787396537Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:45.046752126Z Waiting for performance profile collector pods to become ready: 3 2026-10-31T07:57:45.130679889Z Wrote inspect data to /must-gather/istio. 2026-10-31T07:57:45.687754520Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:57:46.238532414Z Waiting for performance profile collector pods to become ready: 4 2026-10-31T07:57:46.263245575Z Gathering data for ns/openshift-network-console... 2026-10-31T07:57:46.358878129Z Done executing Istio gather script 2026-10-31T07:57:47.265540915Z Gathering data for ns/openshift-console-operator... 2026-10-31T07:57:47.415321880Z Waiting for performance profile collector pods to become ready: 5 2026-10-31T07:57:48.151110620Z Gathering data for ns/openshift-console... 2026-10-31T07:57:48.612197084Z Waiting for performance profile collector pods to become ready: 6 2026-10-31T07:57:49.323550902Z Gathering data for ns/openshift-cluster-storage-operator... 2026-10-31T07:57:49.777545763Z Waiting for performance profile collector pods to become ready: 7 2026-10-31T07:57:50.519081754Z Gathering data for ns/openshift-dns-operator... 2026-10-31T07:57:50.692218668Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:57:50.953297638Z Daemonset perf-node-gather-daemonset ready 3 out of 3 2026-10-31T07:57:51.352689467Z Collecting performance related data for node ip-10-0-140-6.ec2.internal 2026-10-31T07:57:51.360739869Z Collecting performance related data for node ip-10-0-138-89.ec2.internal 2026-10-31T07:57:51.367589855Z Collecting performance related data for node ip-10-0-138-234.ec2.internal 2026-10-31T07:57:51.373571101Z Gathering data for ns/openshift-dns... 2026-10-31T07:57:53.477262885Z Gathering data for ns/openshift-image-registry... 2026-10-31T07:57:54.831448004Z Gathering data for ns/openshift-ingress-operator... 2026-10-31T07:57:55.677313968Z Gathering data for ns/openshift-ingress... 2026-10-31T07:57:55.698977170Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:57:56.590082368Z Gathering data for ns/openshift-ingress-canary... 2026-10-31T07:57:57.719558720Z Gathering data for ns/openshift-insights... 2026-10-31T07:57:59.675771909Z Collecting kubelet logs for node ip-10-0-140-6.ec2.internal 2026-10-31T07:57:59.677545506Z Collecting kubelet logs for node ip-10-0-138-89.ec2.internal 2026-10-31T07:57:59.679182733Z Collecting kubelet logs for node ip-10-0-138-234.ec2.internal 2026-10-31T07:58:00.229985850Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-kj9n5 namespace 2026-10-31T07:58:00.248443505Z INFO: Node performance data collection complete. 2026-10-31T07:58:00.703493780Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:02.245392173Z Gathering data for ns/kserve... 2026-10-31T07:58:05.073012854Z Gathering data for ns/openshift-config... 2026-10-31T07:58:05.707843233Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:05.753818301Z Gathering data for ns/openshift-config-managed... 2026-10-31T07:58:06.603755986Z Gathering data for ns/openshift-kube-apiserver-operator... 2026-10-31T07:58:07.301705164Z Gathering data for ns/openshift-kube-apiserver... 2026-10-31T07:58:08.044720309Z Gathering data for ns/openshift-kube-controller-manager... 2026-10-31T07:58:08.759818219Z Gathering data for ns/openshift-kube-controller-manager-operator... 2026-10-31T07:58:09.535492201Z Gathering data for ns/openshift-kube-scheduler... 2026-10-31T07:58:10.222805036Z Gathering data for ns/openshift-kube-scheduler-operator... 2026-10-31T07:58:10.712297426Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:10.966588380Z Gathering data for ns/openshift-kube-storage-version-migrator... 2026-10-31T07:58:11.909018452Z Gathering data for ns/openshift-kube-storage-version-migrator-operator... 2026-10-31T07:58:12.753221710Z Gathering data for ns/openshift-user-workload-monitoring... 2026-10-31T07:58:13.876889754Z Gathering data for ns/openshift-multus... 2026-10-31T07:58:15.716733405Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:19.036920335Z Gathering data for ns/openshift-ovn-kubernetes... 2026-10-31T07:58:20.720994778Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:23.336945179Z Gathering data for ns/openshift-host-network... 2026-10-31T07:58:24.145661255Z Gathering data for ns/openshift-network-diagnostics... 2026-10-31T07:58:25.726301959Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:25.764066492Z Gathering data for ns/openshift-network-node-identity... 2026-10-31T07:58:26.941852934Z Gathering data for ns/openshift-network-operator... 2026-10-31T07:58:28.041190208Z Gathering data for ns/openshift-cloud-network-config-controller... 2026-10-31T07:58:28.757959945Z Gathering data for ns/openshift-cluster-node-tuning-operator... 2026-10-31T07:58:30.040512868Z Gathering data for ns/openshift-apiserver-operator... 2026-10-31T07:58:30.730838461Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:30.757414952Z Gathering data for ns/openshift-apiserver... 2026-10-31T07:58:31.732001047Z Gathering data for ns/openshift-controller-manager-operator... 2026-10-31T07:58:32.426851454Z Gathering data for ns/openshift-controller-manager... 2026-10-31T07:58:33.148757310Z Gathering data for ns/openshift-cluster-samples-operator... 2026-10-31T07:58:34.451780642Z Gathering data for ns/openshift-operator-lifecycle-manager... 2026-10-31T07:58:35.206032774Z Gathering data for ns/openshift-service-ca-operator... 2026-10-31T07:58:35.736975045Z [disk usage checker] Volume usage percentage: current = 13 ; allowed = 70 2026-10-31T07:58:36.055956267Z Gathering data for ns/openshift-service-ca... 2026-10-31T07:58:37.028924726Z Gathering data for ns/openshift-cluster-csi-drivers... 2026-10-31T07:58:39.225472179Z Wrote inspect data to must-gather. 2026-10-31T07:58:39.225526651Z error: inspection completed with the errors occurred while gathering data: 2026-10-31T07:58:39.225526651Z skipping gathering secrets/support due to error: secrets "support" not found 2026-10-31T07:58:39.445918061Z error: the server doesn't have a resource type "clusters" 2026-10-31T07:58:39.527570594Z Caches written to disk