2026-07-20T11:05:14.312409045Z [disk usage checker] Started
2026-07-20T11:05:14.316272648Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:14.515232984Z Gathering data for ns/openshift-cluster-version...
2026-07-20T11:05:14.640947027Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T11:05:14.721135131Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T11:05:14.801004300Z Gathering data for ns/default...
2026-07-20T11:05:15.106646452Z Gathering data for ns/openshift...
2026-07-20T11:05:15.571511888Z Gathering data for ns/kube-system...
2026-07-20T11:05:16.311853119Z Gathering data for ns/openshift-etcd...
2026-07-20T11:05:16.619673418Z Wrote inspect data to must-gather.
2026-07-20T11:05:16.619709194Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T11:05:16.619709194Z     namespaces "assisted-installer" not found
2026-07-20T11:05:16.917310071Z Waiting on subprocesses to finish execution.
2026-07-20T11:05:16.923657963Z INFO: Gathering on-disk MachineConfig from degraded nodes
2026-07-20T11:05:16.933545433Z Executing Istio gather script
2026-07-20T11:05:16.934321012Z INFO: Gathering machine config daemon's old logs from all nodes
2026-07-20T11:05:16.942354411Z INFO: Collecting host service logs for crio
2026-07-20T11:05:16.942661728Z INFO: Collecting host service logs for kubelet
2026-07-20T11:05:16.943009355Z INFO: Collecting host service logs for rpm-ostreed
2026-07-20T11:05:16.943331988Z INFO: Collecting host service logs for ostree-finalize-staged
2026-07-20T11:05:16.943680448Z INFO: Collecting host service logs for machine-config-daemon-firstboot
2026-07-20T11:05:16.944093762Z INFO: Collecting host service logs for machine-config-daemon-host
2026-07-20T11:05:16.944462387Z INFO: Collecting host service logs for NetworkManager
2026-07-20T11:05:16.944749063Z INFO: Collecting host service logs for openvswitch
2026-07-20T11:05:16.945064069Z INFO: Collecting host service logs for ovs-configuration
2026-07-20T11:05:16.945366409Z INFO: Collecting host service logs for ovsdb-server
2026-07-20T11:05:16.945654111Z INFO: Collecting host service logs for ovs-vswitchd
2026-07-20T11:05:16.946016651Z INFO: Waiting for worker host service log collection to complete ...
2026-07-20T11:05:16.948697516Z WARNING: Collecting one or more kube-apiserver related logs on ALL masters in your cluster. This could take a large amount of time.
2026-07-20T11:05:16.952174220Z INFO: Waiting for node performance related collection to complete ...
2026-07-20T11:05:16.955784236Z INFO: Gathering HAProxy config files
2026-07-20T11:05:17.499565369Z No resources found
2026-07-20T11:05:17.529871615Z INFO: Collecting Insights Archives from insights-operator-555d9d7885-hrwnn
2026-07-20T11:05:17.529871615Z insights-runtime-extractor-dmd4r
2026-07-20T11:05:17.529871615Z insights-runtime-extractor-h6zlg
2026-07-20T11:05:17.529871615Z insights-runtime-extractor-lhh2z
2026-07-20T11:05:17.569028790Z INFO: "kubernetes-nmstate-operator" not detected. Skipping.
2026-07-20T11:05:17.652023274Z No resources found in openshift-etcd namespace.
2026-07-20T11:05:17.742599082Z Wrote inspect data to must-gather.
2026-07-20T11:05:17.753577817Z INFO: Waiting for on-disk MachineConfig collection to complete ...
2026-07-20T11:05:17.753674630Z INFO: on-disk MachineConfig config collection complete.
2026-07-20T11:05:17.892548941Z INFO: "sriov-network-operator" not detected. Skipping.
2026-07-20T11:05:17.902498238Z INFO: namespace openshift-frr-k8s not detected. Skipping.
2026-07-20T11:05:18.006451946Z INFO: "metallb-operator" not detected. Skipping.
2026-07-20T11:05:18.034889843Z INFO: Found 1 replicas - prometheus-k8s-0
2026-07-20T11:05:18.118591111Z error: only SOURCE_DIR and POD:DESTINATION_DIR should be specified as arguments
2026-07-20T11:05:18.118591111Z See 'oc rsync -h' for help and examples
2026-07-20T11:05:18.339912510Z Wrote inspect data to must-gather.
2026-07-20T11:05:18.351214850Z error: the server doesn't have a resource type "performanceprofile"
2026-07-20T11:05:18.523813884Z INFO: Worker host service log collection to complete.
2026-07-20T11:05:18.688385655Z No resources found
2026-07-20T11:05:18.920882860Z INFO: Getting alertmanagers from prometheus-k8s-0
2026-07-20T11:05:19.060394861Z INFO: Waiting for HAProxy config collection to complete ...
2026-07-20T11:05:19.115252373Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T11:05:19.175504923Z Wrote inspect data to must-gather.
2026-07-20T11:05:19.236529272Z error: the server doesn't have a resource type "multi-networkpolicy"
2026-07-20T11:05:19.259110462Z Wrote inspect data to must-gather.
2026-07-20T11:05:19.291630985Z ERROR: No running kube-apiserver pods found
2026-07-20T11:05:19.301457568Z Wrote inspect data to must-gather.
2026-07-20T11:05:19.335355384Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:19.342120114Z Wrote inspect data to must-gather.
2026-07-20T11:05:19.349514453Z INFO: 'previous-logs' folder not found on ip-10-0-135-96.ec2.internal, skipping...
2026-07-20T11:05:19.404154197Z Wrote inspect data to must-gather.
2026-07-20T11:05:19.579027258Z error: the server doesn't have a resource type "machineconfigs"
2026-07-20T11:05:19.634157116Z tar: Removing leading `/' from member names
2026-07-20T11:05:19.638732526Z INFO: HAProxy config collection complete.
2026-07-20T11:05:19.722706490Z INFO: 'previous-logs' folder not found on ip-10-0-136-218.ec2.internal, skipping...
2026-07-20T11:05:19.735361836Z INFO: Getting rules from prometheus-k8s-0
2026-07-20T11:05:19.747982662Z Wrote inspect data to must-gather.
2026-07-20T11:05:19.768222786Z Gathering data for ns/openshift-monitoring...
2026-07-20T11:05:19.816716587Z error: the server doesn't have a resource type "machineconfigpools"
2026-07-20T11:05:19.866079929Z INFO: OLM v1 CRDs not detected. Skipping OLM v1 resource collection.
2026-07-20T11:05:20.012250916Z INFO: 'previous-logs' folder not found on ip-10-0-141-241.ec2.internal, skipping...
2026-07-20T11:05:20.012399095Z INFO: Waiting for Machine Config Daemon termination log collection to complete ...
2026-07-20T11:05:20.012420505Z INFO: Machine Config Daemon termination log collection complete.
2026-07-20T11:05:20.041374817Z Wrote inspect data to must-gather.
2026-07-20T11:05:20.063143470Z Wrote inspect data to must-gather.
2026-07-20T11:05:20.159111804Z INFO: Getting status/config from prometheus-k8s-0
2026-07-20T11:05:20.221125978Z error: the server doesn't have a resource type "kubeletconfigs"
2026-07-20T11:05:20.294316417Z Wrote inspect data to must-gather.
2026-07-20T11:05:20.420520754Z Wrote inspect data to must-gather.
2026-07-20T11:05:20.579347210Z INFO: INTERCONNECT MODE
2026-07-20T11:05:20.579398077Z INFO: Gathering ovn-kubernetes DBs
2026-07-20T11:05:20.596678583Z INFO: Getting status/flags from prometheus-k8s-0
2026-07-20T11:05:20.632009857Z Wrote inspect data to must-gather.
2026-07-20T11:05:20.757570491Z INFO: Gathering OVN_Northbound from ovnkube-node-dqk98...
2026-07-20T11:05:20.757799441Z INFO: Gathering OVN_Northbound from ovnkube-node-l2gzn...
2026-07-20T11:05:20.757986369Z INFO: Gathering OVN_Northbound from ovnkube-node-p252f...
2026-07-20T11:05:20.758235188Z INFO: Gathering OVN_Southbound from ovnkube-node-dqk98...
2026-07-20T11:05:20.758534693Z INFO: Gathering OVN_Southbound from ovnkube-node-l2gzn...
2026-07-20T11:05:20.759057301Z INFO: Gathering OVN_Southbound from ovnkube-node-p252f...
2026-07-20T11:05:20.768971623Z Inspecting resource crd/gatewayclasses.gateway.networking.k8s.io
2026-07-20T11:05:20.902877003Z INFO: Getting status/runtimeinfo from prometheus-k8s-0
2026-07-20T11:05:20.950795821Z Wrote inspect data to must-gather.
2026-07-20T11:05:21.029809603Z tar: Removing leading `/' from member names
2026-07-20T11:05:21.082283556Z tar: Removing leading `/' from member names
2026-07-20T11:05:21.082382239Z tar: Removing leading `/' from member names
2026-07-20T11:05:21.091326465Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:21.096391799Z tar: Removing leading `/' from member names
2026-07-20T11:05:21.103677607Z tar: Removing leading `/' from member names
2026-07-20T11:05:21.115131067Z tar: Removing leading `/' from member names
2026-07-20T11:05:21.227169407Z Wrote inspect data to must-gather.
2026-07-20T11:05:21.242410694Z INFO: Getting targets?state=active from prometheus-k8s-0
2026-07-20T11:05:21.343024038Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:21.346671355Z Inspecting resource crd/gateways.gateway.networking.k8s.io
2026-07-20T11:05:21.455474814Z Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
2026-07-20T11:05:21.491203184Z Wrote inspect data to must-gather.
2026-07-20T11:05:21.520739583Z INFO: Getting status/tsdb from prometheus-k8s-0
2026-07-20T11:05:21.577216278Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:21.599553115Z INFO: Waiting for network log collection to complete ...
2026-07-20T11:05:21.599588962Z INFO: Waiting for ovnk database copies to complete ...
2026-07-20T11:05:21.600112465Z INFO: Copying ovnk databases complete.
2026-07-20T11:05:21.602097674Z 2.9M	must-gather/network_logs/ovnk_database_store
2026-07-20T11:05:21.604374668Z ovnk_database_store/
2026-07-20T11:05:21.604430325Z ovnk_database_store/ovnkube-node-dqk98_sbdb
2026-07-20T11:05:21.626020392Z ovnk_database_store/ovnkube-node-p252f_nbdb
2026-07-20T11:05:21.632508518Z ovnk_database_store/ovnkube-node-p252f_sbdb
2026-07-20T11:05:21.662678526Z ovnk_database_store/ovnkube-node-l2gzn_sbdb
2026-07-20T11:05:21.680379084Z ovnk_database_store/ovnkube-node-dqk98_nbdb
2026-07-20T11:05:21.686945009Z ovnk_database_store/ovnkube-node-l2gzn_nbdb
2026-07-20T11:05:21.698469885Z INFO: Network log collection complete.
2026-07-20T11:05:21.731825169Z Wrote inspect data to must-gather.
2026-07-20T11:05:21.805877541Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:21.808239510Z Inspecting resource crd/grpcroutes.gateway.networking.k8s.io
2026-07-20T11:05:21.954278941Z Wrote inspect data to must-gather.
2026-07-20T11:05:21.961392236Z INFO: Getting status from alertmanager-main-0
2026-07-20T11:05:22.035095052Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:22.124555482Z Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
2026-07-20T11:05:22.234516880Z Error from server (NotFound): deployments.apps "cluster-node-tuning-operator" not found
2026-07-20T11:05:22.236639630Z INFO: Fallback to identify the container image from release info
2026-07-20T11:05:22.242588402Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:22.244879996Z Inspecting resource crd/httproutes.gateway.networking.k8s.io
2026-07-20T11:05:22.324731136Z Gathering data for ns/openshift-network-console...
2026-07-20T11:05:22.455084139Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:22.647678516Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:22.649883206Z Inspecting resource crd/referencegrants.gateway.networking.k8s.io
2026-07-20T11:05:22.736239180Z Gathering data for ns/openshift-console-operator...
2026-07-20T11:05:22.811162458Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:22.990562306Z Wrote inspect data to /must-gather/istio.
2026-07-20T11:05:23.118193390Z Gathering data for ns/openshift-console...
2026-07-20T11:05:23.147783965Z INFO: Image with low level tools to use: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bde4d3e6601b1a43249dcefbf210ae783590d4ff5957380d16c78b4d391ee9eb
2026-07-20T11:05:23.280021496Z daemonset.apps/perf-node-gather-daemonset created
2026-07-20T11:05:23.402033871Z Waiting for performance profile collector pods to become ready: 1
2026-07-20T11:05:23.681805778Z Done executing Istio gather script
2026-07-20T11:05:23.682114253Z Gathering data for ns/openshift-cluster-storage-operator...
2026-07-20T11:05:24.167117263Z Gathering data for ns/openshift-dns-operator...
2026-07-20T11:05:24.340871046Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:24.436432304Z Gathering data for ns/openshift-dns...
2026-07-20T11:05:24.511107834Z Waiting for performance profile collector pods to become ready: 2
2026-07-20T11:05:25.040574925Z Gathering data for ns/openshift-image-registry...
2026-07-20T11:05:25.489625513Z Gathering data for ns/openshift-ingress-operator...
2026-07-20T11:05:25.623264194Z Waiting for performance profile collector pods to become ready: 3
2026-07-20T11:05:25.811070677Z Gathering data for ns/openshift-ingress...
2026-07-20T11:05:26.136109534Z Gathering data for ns/openshift-ingress-canary...
2026-07-20T11:05:26.515723833Z Gathering data for ns/openshift-insights...
2026-07-20T11:05:26.725786611Z Waiting for performance profile collector pods to become ready: 4
2026-07-20T11:05:27.834780055Z Waiting for performance profile collector pods to become ready: 5
2026-07-20T11:05:28.351981477Z Gathering data for ns/openshift-jobset-operator...
2026-07-20T11:05:28.947703393Z Waiting for performance profile collector pods to become ready: 6
2026-07-20T11:05:29.172849735Z Gathering data for ns/openshift-config...
2026-07-20T11:05:29.345558784Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:29.495950175Z Gathering data for ns/openshift-config-managed...
2026-07-20T11:05:29.938456520Z Gathering data for ns/openshift-kube-apiserver-operator...
2026-07-20T11:05:30.069762987Z Waiting for performance profile collector pods to become ready: 7
2026-07-20T11:05:30.221905318Z Gathering data for ns/openshift-kube-apiserver...
2026-07-20T11:05:30.479462470Z Gathering data for ns/openshift-kube-controller-manager...
2026-07-20T11:05:30.771042823Z Gathering data for ns/openshift-kube-controller-manager-operator...
2026-07-20T11:05:31.097293297Z Gathering data for ns/openshift-kube-scheduler...
2026-07-20T11:05:31.193464381Z Daemonset perf-node-gather-daemonset ready 3 out of 3
2026-07-20T11:05:31.374933242Z Gathering data for ns/openshift-kube-scheduler-operator...
2026-07-20T11:05:31.453254781Z Collecting performance related data for node ip-10-0-141-241.ec2.internal
2026-07-20T11:05:31.460224097Z Collecting performance related data for node ip-10-0-135-96.ec2.internal
2026-07-20T11:05:31.467993341Z Collecting performance related data for node ip-10-0-136-218.ec2.internal
2026-07-20T11:05:31.726818050Z Gathering data for ns/openshift-kube-storage-version-migrator...
2026-07-20T11:05:32.053981436Z Gathering data for ns/openshift-kube-storage-version-migrator-operator...
2026-07-20T11:05:32.363497350Z Gathering data for ns/openshift-user-workload-monitoring...
2026-07-20T11:05:32.833901492Z Gathering data for ns/openshift-multus...
2026-07-20T11:05:34.194787103Z Gathering data for ns/openshift-ovn-kubernetes...
2026-07-20T11:05:34.350049458Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:35.555618480Z Gathering data for ns/openshift-host-network...
2026-07-20T11:05:35.845614383Z Gathering data for ns/openshift-network-diagnostics...
2026-07-20T11:05:36.364057714Z Gathering data for ns/openshift-network-node-identity...
2026-07-20T11:05:36.765534073Z Gathering data for ns/openshift-network-operator...
2026-07-20T11:05:37.145321985Z Gathering data for ns/openshift-cloud-network-config-controller...
2026-07-20T11:05:37.478814837Z Gathering data for ns/openshift-cluster-node-tuning-operator...
2026-07-20T11:05:37.966704720Z Gathering data for ns/openshift-apiserver-operator...
2026-07-20T11:05:38.300860910Z Gathering data for ns/openshift-apiserver...
2026-07-20T11:05:38.699756836Z Gathering data for ns/openshift-controller-manager-operator...
2026-07-20T11:05:38.994055698Z Gathering data for ns/openshift-controller-manager...
2026-07-20T11:05:39.255908587Z Gathering data for ns/openshift-cluster-samples-operator...
2026-07-20T11:05:39.355702969Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:39.885216713Z Gathering data for ns/openshift-operator-lifecycle-manager...
2026-07-20T11:05:40.160438604Z Gathering data for ns/openshift-service-ca-operator...
2026-07-20T11:05:40.564433815Z Gathering data for ns/openshift-service-ca...
2026-07-20T11:05:40.905951755Z Gathering data for ns/openshift-cluster-csi-drivers...
2026-07-20T11:05:41.785722485Z Wrote inspect data to must-gather.
2026-07-20T11:05:41.785774356Z error: inspection completed with the errors occurred while gathering data:
2026-07-20T11:05:41.785774356Z     skipping gathering secrets/support due to error: secrets "support" not found
2026-07-20T11:05:44.360636559Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:49.364978864Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:54.369383659Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:05:59.373756348Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:06:04.378231742Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:06:09.382847365Z [disk usage checker] Volume usage percentage: current = 17 ; allowed = 70
2026-07-20T11:06:09.813261514Z Collecting kubelet logs for node ip-10-0-141-241.ec2.internal
2026-07-20T11:06:09.814953478Z Collecting kubelet logs for node ip-10-0-135-96.ec2.internal
2026-07-20T11:06:09.816563925Z Collecting kubelet logs for node ip-10-0-136-218.ec2.internal
2026-07-20T11:06:10.118099452Z daemonset.apps "perf-node-gather-daemonset" deleted from openshift-must-gather-mz5m5 namespace
2026-07-20T11:06:10.123232305Z INFO: Node performance data collection complete.
2026-07-20T11:06:10.254148142Z error: the server doesn't have a resource type "clusters"
2026-07-20T11:06:10.317159827Z Caches written to disk